Free serving software · compute costs
Serve language models through an inference engine designed for throughput and efficient resource use.
The Apache-2.0 serving software is free. Models, GPU hardware, electricity, storage and cloud compute have separate terms and costs. Installing the runtime does not include a hosted inference allowance.
94kstars+ 87930 daysApache-2.0
USE INSTEAD OF
