Request Fine-Tuned Model (OpenAI) v1.0.0 ​

Makes a completion request to the selected fine-tuned model stored in your OpenAI account.

How can I use the Step? ​

The Step lets you create text completions using fine-tuned GPT-3 models. So you can achieve better results for specific tasks related to understanding or generating natural language.

How does the Step work? ​

You authorize the Step, select the appropriate fine-tuned model from your OpenAI account, and provide a text you want to complete. Taking these inputs, the Step requests the fine-tuned model via OpenAI Completions API to generate text completion.

Note: You do not need to provide training examples in a text prompt for fine-tuned models, which saves costs and enables lower latency requests. Read the Fine Tuning guide to learn more.

Prerequisites ​

To get started, make sure you meet the following requirements:

  • You have an OpenAI API key. To retrieve an API key for your requests, visit your API Keys page.
  • You have at least one fine-tuned model stored in your OpenAI account.

Authorization ​

To authorize the Step, you have two options:

  • Inherit from previous Step (default): Use the same authorization as the previous Step in the Flow.
  • Select authorization in the current Step: Choose an existing authorization or create a new one.

In case you need to create a new authorization, follow these steps:

  1. Choose Select authorization in the current Step and then select Create a new authorization from the list.
  2. In the Add authorization modal window, provide the required details:
    • Authorization name: Name your new authorization.
    • API Key: Enter your OpenAI API key.
  3. Click Add to confirm settings and add your new authorization.

Request settings ​

The settings in this section reflect the request body of the Completions API, which you must use as your primary reference:

  • Model: Model to use.
  • Prompt: Text prompt to generate the completion. As you have already tuned the model for the task, you do not need to provide training examples.
  • Max tokens: Controls the maximum length of the generated completion.
  • Temperature: Influences the randomness of the completion.
  • Stop: Defines a sequence where the model should stop generating tokens.
  • Top P: Regulates the diversity of the generated completion.
  • Frequency penalty: Encourages the model to produce unique completions.
  • Presence penalty: Encourages the model to stay on topic.
  • N: The number of completions to generate.
  • Best of: The number of completions to generate and pick the best one from.

Advanced settings ​

In this section, you can specify additional request parameters for your task. All the parameters are optional and refer to the Completions API:

  1. Enable Stream to stream back partial progress.
  2. Enable Echo to include the prompt in the Step response.
  3. For Logit bias, input a JSON object that maps tokens (specified by their token ID in the GPT tokenizer) to an associated bias value from -100 to 100. This way, you modify the likelihood of selected tokens appearing in the completion.
  4. For Logprobs, input the number of most likely tokens to return.

Merge field settings ​

The Step returns the result as a JSON object and stores it in the Merge field variable. Thus you can access the output JSON object from any point of your Flow.

Output example ​

The output contains information about the generated completion and has the following structure:

json
{
  "id": "cmpl-uqkvlQyYK7bGYrRHQ0eXlWi7",
  "object": "text_completion",
  "created": 1589478378,
  "model": "curie:ft-model-name-2023-07-20-19-52-39",
  "choices": [
    {
      "text": "\n\nThis is indeed a test",
      "index": 0,
      "logprobs": null,
      "finish_reason": "length"
    }
  ],
  "usage": {
    "prompt_tokens": 5,
    "completion_tokens": 7,
    "total_tokens": 12
  }
}
{
  "id": "cmpl-uqkvlQyYK7bGYrRHQ0eXlWi7",
  "object": "text_completion",
  "created": 1589478378,
  "model": "curie:ft-model-name-2023-07-20-19-52-39",
  "choices": [
    {
      "text": "\n\nThis is indeed a test",
      "index": 0,
      "logprobs": null,
      "finish_reason": "length"
    }
  ],
  "usage": {
    "prompt_tokens": 5,
    "completion_tokens": 7,
    "total_tokens": 12
  }
}

Error Handling ​

By default, the Handle error toggle is on, and the Step handles errors with a separate exit. If any error occurs during the Step execution, the Flow proceeds down the error exit.

If the Handle error toggle is disabled, the Step does not handle errors. In this case, if any error occurs during the Step execution, the Flow fails immediately after exceeding the Flow timeout. To prevent the Flow from being suspended and continue handling errors, you can place the Flow Error Handling Step before the main logic or your Flow.

Reporting ​

The Step reports once after its execution. You can change the Step log level and add new tags in the section.

Log level ​

By default, the Step inherits its log level from Flow's log level. You can change the Step's log level by selecting an appropriate option from the Log level list.

Tags ​

Tags help organize and filter session information when generating reports. When adding a new tag, you can specify the tag category, label, and value.

Service dependencies ​

  • flow builder - v2.28.3
  • event-manager - v2.3.0
  • deployer - v2.6.0
  • library v2.11.3
  • studio v2.64.1

Release notes ​

v1.0.0

  • Initial release