
Serve language models through an inference engine designed for throughput and efficient resource use.
The Apache-2.0 serving software is free. Models, GPU hardware, electricity, storage and cloud compute have separate terms and costs. Installing the runtime does not include a hosted inference allowance.











