ML Workflows
Inference Workflows
CSSInferenceWorkflow

CSS Inference Workflow

CSS (Closed-Source Software) Workflow is a utility workflow class that has support for various closed-source text-based models. Currently, the following APIs are supported:

  1. OpenAI completions
  2. OpenAI embeddings
  3. Perplexity AI completions
  4. GooseAI completions

Constructor Arguments

  • provider (str): The provider of the model. Currently, the supported providers are OPENAI, PERPLEXITYAI, and GOOSEAI.
  • endpoint (str): This is the model's API endpoint. Currently, the supported endpoints are completions and embeddings.

Additional Installations

Since this workflow uses some additional libraries, you'll need to install infernet-ml[css_inference]. Alternatively, you can install those packages directly. The optional dependencies "[css_inference]" are provided for your convenience.

To install via pip (opens in a new tab):

pip install infernet-ml[css_inference]

Environment Variables

Depending on which provider you use, you need an environment variable named <provider>_APIKEY when running the workflow. For example, if you are using the OpenAI provider, you need to set the OPENAI_APIKEY environment variable.

Input Format

The input format for the CSS Inference Workflow is a dictionary with the following keys:

  • model (str): The name of the model to use. The valid values depend on the CSS model provider. For OpenAI, the valid values are
  • params (Union[CSSCompletionParams, CSSEmbeddingParams]): The parameters associated with the request. It can either be a CSSCompletionParams or a CSSEmbeddingParams.

CSSCompletionParams

  • endpoint (Literal["completions"]): The endpoint of the model. This should be set to "completions".
  • messages (list[ConvoMessage]): A list of Convo messages.

ConvoMessage

  • role (str): Role of the person who the content is attributed to. In the context of completions API's this is either "system" or "user".
  • content (str): The actual content of the convo message.

CSSEmbeddingParams

  • endpoint (Literal["embeddings"]): The endpoint of the model. This should be set to "embeddings".
  • input (str): The input string.

Example

Putting it all together, the following is an example of how to use the CSS Inference Workflow to make a request to the OpenAI's completions API.

Make a file named app.py:

from infernet_ml.workflows.inference.css_inference_workflow import CSSInferenceWorkflow
 
workflow = CSSInferenceWorkflow(provider="OPENAI", endpoint="completions")
workflow.setup()
prompt = "Hey, can shrimp actually fry rice? are you deadass?"
results = workflow.inference(
    {
        "model": "gpt-4-0613",
        "params": {
            "endpoint": "completions",
            "messages": [
                {"role": "system", "content": "You are a helpful assistant."},
                {"role": "user", "content": prompt},
            ],
        },
    }
)
print(f"results: {results}")
BarkHFInferenceWorkflowModel Loader