Chat Templates

Chat templates are a model feature. It's the template with which the model is trained. The template is part of the model extract.

role of the inference engine

The hardness will always communicate with the inference engine with HTTP APIs and data models such as OpenAI. The inference model has to convert the request into the structure mentioned in the chat template.