Q

We are a research lab working on moonshots in deep learning. We are building new learning algorithms that achieve orders-of-magnitude gains in data efficiency and generalization by scaling to unimaginable amounts of compute, even with little data.

Our investors include Jeff Dean, among many others.

Solving generalization

Approximating Solomonoff Induction

Data efficiency, scaling laws & new scaling axes

NanoGPT Slowrun (X)

13x data efficiency with hyper-epoch pretraining

Quadrillion-parameter transformers (X)

Computational depth is all you need:
towards 107-layer neural nets
(X)

Replacing backprop/gradient descent

Dust: Pretraining transformers without
backpropagation
(X)

Noise: Speedrun for zero-order optimization

Applications

Towards universal data-efficient pretraining

reach out: research@qlabs.sh