LakeQL
Overview
  • Introduction
  • Datasource Contract
  • API Reference
Writing
  • Write Strategies
  • Load Strategies
  • Partitioning
Storage
  • Storage Operations
Trino
  • Connection
  • Write Config
  • Metadata
  • API Reference
Fabric
  • Connection
  • Metadata
  • API Reference
GitHub
LakeQL
  1. Data Sources
  2. Overview
  3. API Reference

On this page

  1. API Reference
    1. TAdapterName
    2. ISourceConfig
    3. TDataSourceAuth
    4. TResolveAuth
    5. IDataSourceCapabilities
    6. IDataSource
    7. IColumnInfo
    8. IQueryProps
    9. ITableDefinition
    10. TPrimitiveType
    11. ITypeMapper
    12. IDataSourceReadInput
    13. IDataSourceReadResult
    14. IDataSourceReadStrategy
    15. IReadStrategyPaging
    16. IDataSourceWriteConfig
    17. IDataSourceWriteInput
    18. IDataSourceWriteStrategy
    19. TLoadStrategy
    20. IAdapterFactory
    21. CAuthenticationError
    22. CCancellationError
    23. CDataSourceError
    24. TDataSourceErrorCategory
    25. CDataSourceValidationError
    26. CNetworkError
    27. CQueryExecutionError
    28. CTimeoutError
    29. CUnsupportedOperationError
    30. CUnsupportedWriteStrategy
      1. write
    31. FcreateTypeMapper
    32. ITypeMapperTables
    33. VDEFAULT_MAX_PER_PAGE
    34. FvalidatePaging
    35. IReadValidationConfig
    36. FescapeLiteral
    37. FisPlainObject
    38. FquoteQualified
    39. FrejectControlChars

API Reference

Complete reference for the @lakeql/datasource contract package.

The complete public API of @lakeql/datasource, generated from the package's TypeScript source. This is the adapter-neutral contract package. It defines the interfaces every adapter implements.

The core contracts are DataSource, DataSourceReadStrategy, DataSourceWriteStrategy, and AdapterFactory. The error classes, rooted at DataSourceError, are the normalized errors adapters throw. The API layer maps them to GraphQL responses.

Open adapter name type.

Adapter identifiers are plain strings so that third-party adapters do not require changes to a central union type.

Definition

string

Shared source configuration that identifies a table within a data source without coupling to adapter-specific connection details.

Properties

NameType
dataSourceRequiredstring

Runtime name of the configured data source.

adapterRequiredAdapterName

Open identifier of the adapter that owns this source.

catalogRequiredstring

Source catalog containing the target table.

schemaRequiredstring

Source schema containing the target table.

tableNameRequiredstring

Name of the target table.

Neutral authentication types for data source access.

These types live in @lakeql/datasource (design decision 4) because they are required by the adapter factory and read strategy contracts.

Discriminated union of authentication credentials that can be transported through the core without adapter-specific knowledge.

The set of variants may be extended by a future auth design, but the core must carry request auth without interpreting adapter-level credential details.

Definition

| {
      /** Username and password credentials. */
      type: "basic"
      /** Username sent to the data source. */
      username: string
      /** Password sent to the data source. */
      password: string
    }
  | {
      /** Static bearer token credentials. */
      type: "bearer"
      /** Token sent in the authorization header. */
      token: string
    }
  | {
      /** OAuth 2.0 access-token credentials. */
      type: "oauth2"
      /** Current OAuth 2.0 access token. */
      accessToken: string
    }

Auth resolution function injected into read strategies and adapter factories. Given a data source name and an opaque request context, it returns optional credentials and adapter-level options.

Definition

(
  dataSourceName: string,
  context: unknown
) => Promise<{
  /** Credentials to attach to the data-source request. */
  requestAuth?: DataSourceAuth
  /** Adapter-owned request options. */
  options?: Record<string, unknown>
}>
See:DataSourceAuth

Describes actual support for operations, metadata discovery, write modes, and DDL provided by a data source.

Adapters populate this at construction time so that the runtime can avoid selecting unsupported operations.

Properties

NameType
queryRequiredboolean

Whether the data source supports query execution.

streamRequiredboolean

Whether the data source supports streaming results.

metadataRequiredObject

Metadata discovery support per catalog object type.

PropertyType
catalogsboolean

Whether catalog discovery is supported.

schemasboolean

Whether schema discovery is supported.

tablesboolean

Whether table discovery is supported.

viewsboolean

Whether view discovery is supported.

columnsboolean

Whether column discovery is supported.

writeRequiredObject

Write support and available write modes.

PropertyType
supportedboolean
modesstring[]

Available write modes — an open string array so that future adapters can provide modes not known to the core.

ddlRequiredObject

DDL support.

PropertyType
createTableboolean

Whether creating tables is supported.

dropTableboolean

Whether dropping tables is supported.

Adapter-neutral data source contract.

Every adapter package implements this interface to provide query execution, streaming, metadata discovery, DDL, and lifecycle management without coupling the core to adapter-specific details.

Properties

NameType
adapterRequiredAdapterName

Open adapter identifier for this data source instance.

capabilitiesRequiredDataSourceCapabilities

Capabilities supported by this data source.

typeMapperRequiredTypeMapper

Maps between this adapter’s engine types and LakeQL primitive types.

queryRequired<TRow>(props: QueryProps) => Promise<TRow[]>

Executes a SQL query and returns all matching rows.

streamRequired<TRow>(props: QueryProps) => Promise<AsyncGenerator<TRow>>

Executes a SQL query and returns its rows as an asynchronous generator.

catalogsRequired() => Promise<string[]>

Lists catalogs available to the current data source.

schemasRequired(catalog: string) => Promise<string[]>

Lists schemas within a catalog.

tablesRequired(catalog: string, schema: string) => Promise<string[]>

Lists tables within a catalog and schema.

viewsRequired(catalog: string, schema: string) => Promise<string[]>

Lists views within a catalog and schema.

columnsRequired( catalog: string, schema: string, table: string ) => Promise<ColumnInfo[]>

Lists column metadata for a table.

createTableRequired(definition: TableDefinition) => Promise<void>

Creates a table from a neutral definition.

dropTableRequired(catalog: string, schema: string, table: string) => Promise<void>

Drops a table.

qualifiedTableNameRequired(config: SourceConfig) => string

Builds an adapter-specific fully qualified table name.

closeRequired() => Promise<void>

Releases all resources held by this data source.

Metadata describing a single column in a table.

Properties

NameType
nameRequiredstring

Column name as exposed by the data source.

typeRequiredstring

Native or normalized data type name.

extra?string

Additional adapter-provided metadata, such as precision or nullability.

description?string

Optional human-readable column description.

Parameters for executing a SQL query against a data source.

Properties

NameType
sqlRequiredstring

SQL statement to execute.

signal?AbortSignal

Optional cancellation signal for the query.

requestAuth?DataSourceAuth

Credentials resolved for this request.

options?Record<string, unknown>

Adapter-specific query options.

Definition for creating a new table.

Properties

NameType
catalogRequiredstring

Catalog in which to create the table.

schemaRequiredstring

Schema in which to create the table.

tableRequiredstring

Name of the table to create.

columnsRequiredObject[]

Ordered column definitions for the new table.

PropertyType
namestring
typestring

Adapter-neutral type mapping between engine-specific data types and the canonical LakeQL primitive type system.

The canonical LakeQL primitive types used across codegen and metadata.

This union mirrors PrimitiveType in @lakeql/schema-generator (the source of truth). It is duplicated here on purpose: this contract package must not depend on an implementation package. Keep the two in sync.

Definition

| "String"
  | "Integer"
  | "Float"
  | "Boolean"
  | "Date"
  | "DateTime"

Maps between an adapter’s engine-specific data types and LakeQL PrimitiveTypes.

Each adapter provides its own implementation that understands its engine’s native type system (Trino, T-SQL, etc.).

Properties

NameType
toPrimitiveTypeRequired(engineType: string) => PrimitiveType

Converts an engine-specific type string to a LakeQL PrimitiveType.

Implementations SHALL normalize the input (case, type parameters) and fall back to "String" when the engine type has no known mapping.

toEngineTypeRequired(primitiveType: PrimitiveType) => string

Converts a LakeQL PrimitiveType to the engine’s native type string.

Shared read input that types common values while leaving filter, schema, and context generic for adapter- or API-specific extensibility.

Type Parameters

ParameterConstraintDefault
TFilter—unknown
TSchema—unknown
TContext—unknown

Request-scoped context passed through to the read implementation.

Properties

NameType
sourceConfigRequiredSourceConfig

Table and adapter coordinates for the requested source.

selectFieldsRequiredreadonly string[]

Source fields to include in the result.

filterRequiredTFilter

Adapter- or API-specific filter expression.

pagingRequiredObject

One-based page number and the requested page size. perPage is optional; the strategy substitutes the resolved default when omitted.

PropertyType
pagenumber
perPage?number | null
sortingRequiredreadonly unknown[]

Adapter- or API-specific sort definitions.

transformFieldsRequiredReadonly<Record<string, string>>

Mapping from source field names to their result aliases.

dateFieldsRequiredreadonly string[]

Fields that require date-specific handling by the adapter.

schemaRequiredTSchema

Schema or selection metadata required by the read implementation.

contextRequiredTContext

Request-scoped context passed through without core interpretation.

Neutral read result exposing total count, page information, and typed result nodes.

Type Parameters

ParameterConstraintDefault
T——

Shape of each record in the result page.

Properties

NameType
totalCountRequirednumber

Total number of matching records across all pages.

pageInfoRequiredObject

Paging metadata for the returned result page.

PropertyType
maxPagesnumber

Total number of available pages, or zero when no records match.

hasNextboolean

Whether another page is available after this one.

hasPreviousboolean

Whether a page is available before this one.

currentPagenumber

One-based number of the returned page.

nextPagenumber | null

One-based number of the next page, or null when on the last page.

previousPagenumber | null

One-based number of the previous page, or null when on the first page.

nodesRequiredT[]

Typed records returned for the requested page.

Strategy contract for reading data from a data source. Adapter-specific read implementations conform to this interface.

Properties

NameType
readRequired<T>(input: DataSourceReadInput) => Promise<DataSourceReadResult<T>>

Reads a page of records. Implementations must validate paging before building or executing their query.

Resolved per-data-source paging limits.

Owned by @lakeql/datasource so resolved config values flow to the adapter factory and read strategies with compile-time assignability.

Properties

NameType
defaultPerPageRequirednumber

Page size used when a request omits perPage.

maxRecordsPerPageRequirednumber

Upper bound a request’s perPage may not exceed.

Shared write configuration containing table coordinates and optional common write settings. Storage-backed strategies may require basePath; direct-insert strategies omit it.

Properties

NameType
tableRequiredObject

Catalog, schema, and table name of the write target.

PropertyType
catalogstring
schemastring
tableNamestring
loadStrategy?LoadStrategy

Behavior to apply when records reach an existing target.

basePath?string

Storage base path required by storage-backed write strategies.

keepShadowTables?boolean

When true, instructs strategies that manage shadow tables to retain them after the write completes. Useful for debugging or auditing intermediate state.

writeId?string

Reserved for future idempotency support. Accepting this value now ensures forward-compatible configuration, but it has no current behavioral effect.

partitioning?unknown

Adapter-owned partitioning configuration.

partitioningFormat?string

Adapter-owned format for partition values.

Schema-library-agnostic write input containing records, a generic schema value, and write configuration.

Properties

NameType
recordsRequiredRecord<string, unknown>[]

Records to write to the configured target.

schemaRequiredunknown

Schema understood by the adapter-owned write implementation.

configRequiredDataSourceWriteConfig

Common and adapter-owned settings for this write.

Strategy contract for writing data to a data source. Adapter-specific write implementations conform to this interface.

Properties

NameType
writeRequired(input: DataSourceWriteInput) => Promise<void>

Writes the supplied records and completes when the write has finished.

Supported load strategies for write operations.

  • full_load — replace the target table completely
  • full_load_append — replace existing partitions; append new ones
  • append — add records without removing existing data

Definition

"full_load" | "full_load_append" | "append"

Adapter factory contract.

Every adapter package exposes a single factory that the runtime uses to construct a data source, read strategy, and write strategy without adapter-specific switch statements.

Config parameters use unknown to keep the core free of adapter-specific type imports. Adapter factories own validation and narrowing of their configuration objects.

Environment-based connection configuration is NOT part of this contract. If an adapter needs environment parsing, it provides a separate config-layer provider that feeds the central config loader.

Properties

NameType
adapterNameRequiredAdapterName

Unique adapter identifier (e.g. "trino", "fabric").

createDataSourceRequired(config: unknown, name: string) => DataSource

Constructs an adapter-owned data source from validated adapter-specific configuration and its configured runtime name.

createReadStrategyRequired(args: { dataSource: DataSource resolveAuth: ResolveAuth paging?: ReadStrategyPaging }) => DataSourceReadStrategy

Constructs a read strategy with its data source and auth resolver.

createWriteStrategyRequired(args: { dataSource: DataSource dataSourceName: string writeConfig: unknown }) => DataSourceWriteStrategy

Constructs a write strategy from adapter-owned configuration.

Operational error for authentication or authorization failures.

Fixes category to "auth".

Constructor

NameType
messageRequiredstring

Human-readable explanation of the auth failure. Must not contain credentials.

adapterRequiredAdapterName

Open identifier of the adapter that produced this error.

dataSourceRequiredstring

Runtime name of the data source that produced this error.

options?{ cause?: unknown }

Optional lower-level cause to preserve.

Operational error for explicitly cancelled operations.

Fixes category to "cancelled".

Constructor

NameType
messageRequiredstring

Human-readable explanation of the cancellation. Must not contain credentials.

adapterRequiredAdapterName

Open identifier of the adapter that produced this error.

dataSourceRequiredstring

Runtime name of the data source that produced this error.

options?{ cause?: unknown }

Optional lower-level cause to preserve.

Base operational error for datasource failures.

Every instance carries a fixed category, the adapter name and dataSource identifier that produced the failure, and optionally the lower-level cause.

Adapter-owned configuration and connection errors remain the adapter’s responsibility — they should extend their own error types, not this one.

Constructor

NameType
messageRequiredstring

Human-readable explanation of the failure. Must not contain credentials.

adapterRequiredAdapterName

Open identifier of the adapter that produced this error.

dataSourceRequiredstring

Runtime name of the data source that produced this error.

categoryRequiredDataSourceErrorCategory

Fixed category for this error class.

options?{ cause?: unknown }

Optional lower-level cause to preserve.

Properties

NameType
adapterRequiredAdapterName

Open identifier of the adapter that produced this error.

dataSourceRequiredstring

Runtime name of the data source that produced this error.

categoryRequiredDataSourceErrorCategory

Fixed category classifying this operational error.

Category strings that classify operational datasource errors.

Each category maps to a specific DataSourceError subclass, allowing consumers to branch on the category without instanceof checks or message parsing.

Definition

| "auth"
  | "query"
  | "network"
  | "timeout"
  | "cancelled"
  | "unsupported"

Validation error for core contract values.

Identifies the invalid field and value when the validation logic can reasonably determine them. Field identification is optional.

Constructor

NameType
messageRequiredstring

Human-readable explanation of the invalid value.

options?{ field?: string; value?: unknown; cause?: unknown }

Optional field, value, and lower-level cause metadata.

Properties

NameType
field?string

Name of the invalid field, when validation can identify one.

value?unknown

Invalid value, when validation can safely expose it.

Operational error for network-level failures (connection refused, DNS resolution, TLS handshake, etc.).

Fixes category to "network".

Constructor

NameType
messageRequiredstring

Human-readable explanation of the network failure. Must not contain credentials.

adapterRequiredAdapterName

Open identifier of the adapter that produced this error.

dataSourceRequiredstring

Runtime name of the data source that produced this error.

options?{ cause?: unknown }

Optional lower-level cause to preserve.

Operational error for query execution failures.

Fixes category to "query".

Constructor

NameType
messageRequiredstring

Human-readable explanation of the query failure. Must not contain credentials.

adapterRequiredAdapterName

Open identifier of the adapter that produced this error.

dataSourceRequiredstring

Runtime name of the data source that produced this error.

options?{ cause?: unknown }

Optional lower-level cause to preserve.

Operational error for operations that exceeded a time limit.

Fixes category to "timeout".

Constructor

NameType
messageRequiredstring

Human-readable explanation of the timeout. Must not contain credentials.

adapterRequiredAdapterName

Open identifier of the adapter that produced this error.

dataSourceRequiredstring

Runtime name of the data source that produced this error.

options?{ cause?: unknown }

Optional lower-level cause to preserve.

Operational error for operations that an adapter does not support.

Fixes category to "unsupported" and exposes the rejected operation name.

Constructor

NameType
messageRequiredstring

Human-readable explanation. Must not contain credentials.

adapterRequiredAdapterName

Open identifier of the adapter that produced this error.

dataSourceRequiredstring

Runtime name of the data source that produced this error.

operationRequiredstring

Name of the unsupported operation.

options?{ cause?: unknown }

Optional lower-level cause to preserve.

Properties

NameType
operationRequiredstring

Name of the unsupported operation (e.g. "write").

A write strategy that rejects every write with an UnsupportedOperationError.

Adapter factories use this to satisfy the DataSourceWriteStrategy contract when the adapter does not yet provide a real write path.

Constructor

NameType
optionsRequired{ adapter: AdapterName; dataSource: string }

Adapter and data-source identifiers attached to the

  • rejection error.

Properties

NameType
#adapterRequiredAdapterName
#dataSourceRequiredstring

Methods

write

Always rejects with an UnsupportedOperationError.

The input is intentionally ignored — the strategy does not inspect, destructure, validate, log, mutate, forward, or otherwise access it.

Parameters
NameType
_inputRequiredDataSourceWriteInput

Write input (unused).

Returns
Promise<void>

A promise that rejects with an UnsupportedOperationError.

Builds a TypeMapper from an adapter’s engine↔primitive lookup tables.

The returned toPrimitiveType normalizes its input before the lookup and falls back to "String" for unknown engine types. toEngineType reads the mapping for the closed PrimitiveType union.

Parameters

NameType
tablesRequiredTypeMapperTables

The adapter’s bidirectional type-mapping tables.

Returns

TypeMapper

A TypeMapper backed by the supplied tables.

Lookup tables that define an adapter’s type mapping in both directions.

Properties

NameType
toPrimitiveRequiredRecord<string, PrimitiveType>

Maps a normalized engine type (lowercased, without parameters) to a LakeQL PrimitiveType. Unknown engine types fall back to "String".

toEngineRequiredRecord<PrimitiveType, string>

Maps each LakeQL PrimitiveType to the engine’s native type string.

Default upper bound for perPage when no ReadValidationConfig override is supplied.

Definition

10_000

Validates paging (and optional sorting) parameters before query building.

Read-strategy implementations MUST call this before building or executing any query. Invalid input throws a DataSourceValidationError to block query construction.

Rules:

  • page : finite integer ≥ 1.
  • perPage : finite integer between 1 and config.maxPerPage (default {@link DEFAULT_MAX_PER_PAGE}).
  • sorting (when present): must be an array.

Parameters

NameType
pagingRequiredObject

One-based page number, page size, and optional sorting.

PropertyType
pagenumber
perPagenumber
sorting?readonly unknown[]
config?ReadValidationConfig

Optional validation limits (e.g. maxPerPage).

Configuration for read-input validation.

Properties

NameType
maxPerPage?number

Maximum allowed perPage value.

Default: 10000

Escape a value as a SQL string literal.

The input is treated as a string literal value. Embedded single-quote characters are escaped by doubling them (' → ''), and the result is wrapped in single quotes. The escaping is identical across the Trino and Fabric dialects.

Parameters

NameType
valueRequiredstring

The raw string value to escape.

Returns

string

The safely single-quoted literal string. *

Returns true if value is a non-null plain object: its prototype is Object.prototype or null, and it is not an array.

This is the strict variant shared across all adapter modules. It rejects class instances, Date, RegExp, and other exotic objects, and it is safe for prototype-less objects created via Object.create(null).

Parameters

NameType
valueRequiredunknown

The value to test.

Returns

value is Record<string, unknown>

true if value is a plain object, otherwise false.

Quote a fully-qualified identifier composed of one or more segments.

Applies the dialect-specific quoteFn independently to each segment and joins them with literal dots. Requires at least one segment. The per-segment quoting semantics are owned by the caller’s quoteFn.

Parameters

NameType
quoteFnRequired(segment: string) => string

Dialect-specific single-segment quoter (e.g. double-quote or bracket).

segments?string[]

One or more raw identifier segments (e.g. catalog, schema, table).

Returns

string

The safely quoted qualified identifier string. *

Throws if value contains any C0 (U+0000–U+001F) or C1 (U+007F–U+009F) control character.

Parameters

NameType
valueRequiredstring

The raw string to validate.

contextRequiredstring

Human-readable label used in the thrown error message.

Previous page

Datasource Contract

Next page

Writing