stubb

Stubb is a library for creating and evaluating structured outputs from language models.

Installation

Install this library using pip:

pip install stubb

Usage

Usage instructions go here.

Other Libraries

Stubb is similar in spirit to several other libraries, including:

It is different in the following ways:

Instructor uses the OpenAI API, and uses the function calling to generate json. Stubb is similar, but uses constrained decoding via a grammar to generate json, which is compatible with many open source language models.

Guardrails uses a custom xml specification, (.rail), as well as Pydantic models to generate json. Stubb is similar, but has a much simpler API and tries to do "less things". Both Instructor and Guardrails support re-running model APIs based on complex post-validation (i.e not purely a schema). Stubb does not do this.

Outlines takes a similar approach to to Stubb to generate valid Pydantic models, but it uses regex parsing to do constrained decoding. This is written in Python and accelerated, during sampling, with Numba. In constrast, Stubb uses a GBNF grammar, which is supported in C++ directly via the llama.cpp project.

Stubb is designed to be simple to use, and not to do "too much for you". It uses validated technologies like Pydantic, FastAPI, Sqlite, and is modular - you can use the components you want, and not the ones you don't.

Development

To contribute to this library, first checkout the code. Then create a new virtual environment:

cd stubb
python -m venv venv
source venv/bin/activate

Now install the dependencies and test dependencies:

pip install -e '.[test]'

To run the tests:

pytest

Watch for Ollama to merge one of these PRs.

https://github.com/ollama/ollama/pulls?q=is%3Apr+is%3Aopen+grammar