Wvlet, pronounced as weave-let, is a new cross-SQL flow-style query language for functional data modeling and interactive data exploration. Wvlet works with various types of SQL-based database engines, including DuckDB, Trino, Hive, etc.

• Documentation
• Milestones
• Project Roadmap

Wvlet queries (saved as .wv files) provide a natural way to describe data processing pipelines, which will eventually be compiled into a sequence of SQL queries. While SQL is a powerful language for processing data, its syntax often does not match the semantic order of data processing. Let's see the following example: The syntactic order of SQL's SELECT ... statements mismatches with the actual data flow inside the SQL engines (cited from A Critique of Modern SQL And A Proposal Towards A Simple and Expressive Query Language (CIDR '24)):

For overcoming this shortcoming of SQL, Wvlet starts from a table scan statement from ..., and the result can be streamlined to the next processing operators like where, group by, select, etc., as if passing table values through a pipe to the next operator:

from (table)
where (filtering condition)
...
where (more filtering condition can be added)
group by (grouping keys, ...)
where (group condition can be added just with where)
select (columns to output)
order by (ordering columns...)
limit (limiting the number of output rows)

With this flow style, you can describe data processing pipelines in a natural order to create complex queries. You can also add operators for testing or debugging data in the middle of the query. This flow syntax is gaining traction and has been adopted in Google's SQL to simplify writing SQL queries. For more details, see SQL Has Problems. We Can Fix Them: Pipe Syntax In SQL (VLDB 2024).

• Flow-style syntax: Write queries in the natural order of data processing
• Multi-database support: Works with DuckDB, Trino, Hive, and more
• Static catalog support: Compile queries offline without database connections
• Interactive REPL: Explore data interactively with auto-completion
• Type-safe queries: Catch errors at compile time with schema validation
• Modular queries: Organize and reuse queries as functions

Wvlet can export database catalog metadata (schemas, tables, columns) to JSON files and use them for offline query compilation. This enables:

• CI/CD Integration: Validate queries in pull requests without database access
• Faster Development: Compile queries instantly without network latency
• Version Control: Track schema changes alongside your queries

# Export catalog metadata
wvlet catalog import --name mydb

# Compile queries offline
wvlet compile query.wv --use-static-catalog --catalog mydb

See Catalog Management for more details.

Many thanks to everyone who has contributed to our progress: