
Run language model inference across different hardware with a portable C and C++ engine.
The MIT-licensed inference runtime has no software license fee. Supply suitable hardware and check each model license. Rented GPU or server resources and unrelated hosted services can cost extra.





