LakeQL
Overview
  • Installation
Commands
  • init
  • pull
  • create-registry
  • create-endpoint
  • generate-import-config
  • list-schemas
  • list-tables
  • list-views
  • list-columns
  • add
Configuration
  • Environment Variables
  • Config File
GitHub
LakeQL
  1. CLI
  2. Commands
  3. pull

On this page

  1. Bulk pull
  2. Options
    1. --source <name>
    2. --catalog <catalog>
    3. --type <type>
    4. --schema <schema>
    5. --table <table>
    6. --skip-registry
    7. --source-path <path>
    8. --concurrency <count>
    9. --bulk
    10. --bulk-config <path>

pull

Interactively generate a query endpoint from a remote table.

Run pull to introspect a remote table and generate its typed endpoint files. It writes the interface, query schema, config, and JSON schema. Run it with no options and it prompts for the source, catalog, schema, and table.

pnpm lakeql-cli pull

Pass those values as options to skip the prompts:

pnpm lakeql-cli pull --source warehouse --catalog hive --schema sales --table
orders

Generated files land under your configured source path, organized by data source, catalog, and schema. See Key Concepts for the layout. To generate many endpoints at once, use --bulk with a bulk config.

Bulk pull #

pull --bulk reads a bulk config (import.config.mjs by default, generated by generate-import-config) and pulls every entry. Each entry may name the dataSource it belongs to:

pnpm lakeql-cli pull --bulk
  • Without --source , every entry is processed and each is resolved against its own dataSource . A multi-source config pulls from all of its sources in one run; each distinct source is connected once and closed at the end.
  • With --source <value> , --source acts as a filter: only entries whose dataSource matches are pulled (the others are skipped).

In a single-source project (exactly one data source configured) you may omit dataSource on each entry — it defaults to the sole source. In a multi-source project, dataSource is required per entry; an entry that omits it when more than one source is configured (and no --source filter pins one) is a configuration error.

A hand-authored entry may list views instead of or in addition to tables; bulk pull generates endpoints for views the same way it does for tables, and each entry must have at least one non-empty tables or views list. (generate-import-config only ever emits tables, so add any views by hand.)

1
2
3
4
5
6
7
8
9
10
11
12
// import.config.mjs
/** @type {import('@lakeql/cli').BulkPullConfig} */
export default [
  {
    dataSource: "warehouse",
    catalog: "hive",
    schema: "sales",
    tables: ["orders"],
    views: ["orders_summary"],
  },
]

pull always generates a read-only endpoint (mutation: false). It does not add a Mutation root field or otherwise make the endpoint writable. To enable writes, add a mutation block to the generated endpoint.json and regenerate. See Mutations.

pull recreates the endpoint's output directory. It recursively deletes the target table's generated folder before writing. Do not keep hand-edited files inside a generated endpoint directory. Put customizations in your own modules.

pull updates the config registry by default, so new endpoints are type-safe. Pass --skip-registry to skip that step, then run create-registry later.

--source <name>

Name of the configured data source to use

PropertyValue
Typestring
RequiredNo

--catalog <catalog>

catalog to use

PropertyValue
Typestring
RequiredNo

--type <type>

Show tables or views

PropertyValue
Typestring
RequiredNo

--schema <schema>

schema to use

PropertyValue
Typestring
RequiredNo

--table <table>

table to use

PropertyValue
Typestring
RequiredNo
Default[]

--skip-registry

Skip registry update

PropertyValue
Typeboolean
RequiredNo
Defaultfalse

--source-path <path>

Base path for generated code (resolved from the command invocation directory). Files are created in schemas/generated|custom inside this path.

PropertyValue
Typestring
RequiredNo
Defaultcommand invocation directory

--concurrency <count>

Maximum number of concurrent pull operations for multi-item pulls.

PropertyValue
Typestring
RequiredNo
Default8

--bulk

Run in bulk mode using a config file

PropertyValue
Typeboolean
RequiredNo
Defaultfalse

--bulk-config <path>

Path to the bulk import config file (default: import.config.{mjs,ts,js,json})

PropertyValue
Typestring
RequiredNo

Previous page

init

Next page

create-registry