Huggingface Inference Client Workflow

This workflow uses Huggingface Inference Client library (opens in a new tab) to run all models that are hosted on Huggingface Hub.

Constructor Arguments

model_name (str) : The model name from Huggingface Hub.
task (str): The task to be performed by the model. Refer below for the supported tasks.

Supported Tasks

"text_generation"
"text_classification"
"token_classification"
"summarization"

Input Format

`text_generation`

For text_generation task, the input format is:

{
    "prompt": str,
    "details": bool,
    "stream": bool,
    "do_sample": bool,
    "max_new_tokens": int,
    "best_of": Optional[int],
    "repetition_penalty": Optional[float],
    "return_full_text": bool,
    "seed": Optional[int],
    "stop_sequences": Optional[str],
    "temperature": Optional[float],
    "top_k": Optional[int],
    "top_p": Optional[float],
    "truncate": Optional[int],
    "typical_p": Optional[float],
    "watermark": bool,
    "decoder_input_details": bool
}

`text_classification`

For text_classification task, the input format is:

{
    "text": str
}

`token_classification`

For token_classification task, the input format is:

{
    "text": str
}

`summarization`

For summarization task, the input format is:

{
    "text": str,
    "parameters": Optional[dict[str, Any]]
}

Example

from infernet_ml.workflows.inference.hf_inference_client_workflow import HFInferenceClientWorkflow
 
workflow = HFInferenceClientWorkflow(task="text_classification", model="Kaludi/Reviews-Sentiment-Analysis")
workflow.setup()
results = workflow.inference({
    "text": "I love this product, going to buy one for my friend too!"
})
print(f"results: {results}")

Architecture ONNXInferenceWorkflow