Skip to content

Latest commit

 

History

History
581 lines (438 loc) · 20.4 KB

tutorial.md

File metadata and controls

581 lines (438 loc) · 20.4 KB

Tutorials

Preparation

Register API (Required)

This add-on requires an API key to access an OpenAI API.
You need to create an OpenAI account and get an API key from the OpenAI Official Site.

Note that you need to pay to use an OpenAI API.
Before using this add-on, check the Pricing Page.

Once you have got an API key, enter the OpenAI API key in the add-on preferences (a).
Click the [Check API Connection] button (b) and check if the status is OK (c).
If the status is not OK, see the error message and solve the issue.
If you are behind the proxy, input [HTTP Proxy] and [HTTPS Proxy].

Add-on Preferences

You can change the other properties in the add-on preferences.

Enable Audio Input (Optional)

Code Tool supports the audio input.
To enable the audio input, click [Enable Audio Input] button on the category at the add-on preferences.

Enable Audio Input

Usage Statistics

You can see the statistics for the usage of the OpenAI API.

Note: These statistics are not perfect for now. We recommend you to check the usage detail on the OpenAI website.

Usage Statistics

Add-on Preferences

The add-on preferences allows you to change the behavior of the operators.

Category Property Description
System Popup Menu Width Width of the popup menu from workspace tools.
Async Execution Execute operations asynchronously.
Show Status Show request status.
Audio Tool Audio Tool Model Model to be used for Audio Tool.
(Supported Model: whisper-1)
Chat Tool Chat Tool Model Model to be used for Chat Tool.
(Supported Model: gpt-3.5-turbo, gpt-4, gpt-4-32k)
Wrap Width Wrap width of the chat tool log
Code Tool Code Tool Model Model to be used for Code Tool.
(Supported Model: gpt-3.5-turbo, gpt-4, gpt-4-32k)
Audio Input Configuration > Language Language for the audio input in the code tool.
This property is enabled when the audio input is enabled.
Recording Configuration > Format Formats for the audio recording.
This property is enabled when the audio input is enabled.
Recording Configuration > Channels Channels for the audio recording.
This property is enabled when the audio input is enabled.
Recording Configuration > Rate Sampling rate for the audio recording.
This property is enabled when the audio input is enabled.
Recording Configuration > Chuck Size Frames per buffer.
This property is enabled when the audio input is enabled.
Recording Configuration > Silence Threshold Threshold to stop the audio recording.
This property is enabled when the audio input is enabled.
Recording Configuration > Silence Duration Limit The seconds to stop the audio recording.
This property is enabled when the audio input is enabled.

Image Tool

Image Tool can generate/edit an image.
Image Tool can be accessed from [OpenAI] tab on the Sidebar Panel or from the tool [OpenAI Image Tool] in Image Editor.

Generate Images

There are 2 ways to generate an image from a prompt.

1. From Sidebar Panel

Generate Images from Sidebar Panel 1

  1. Go to Image Editor.
  2. Open [OpenAI] tab on Sidebar.
  3. Open a [Generate Image] sub-panel.
  4. Change the properties.
  5. Enter a prompt. (a)
  6. Click the [▶] button. (b)
  7. The generated image appears in the [Generated Images] sub-panel and loaded into the Image Editor. (c)

Note: The generated image will be saved in <addon_folder>/_data/image/genereated.

Properties
Size The size of the images to generate.
Num The number of images to generate.
Name Name for a generated image.
If [Auto] is checked, the name will be generated automatically.

You can load a generated image into Blender or parmanently remove it from a file.

Generate Images from Sidebar Panel 2

2. From Tool

Generate Images from Tool

  1. Go to Image Editor.
  2. Select the tool [OpenAI Image Tool]. (a)
  3. Press the 'SPACE' key in the Image Editor, and the pop-up dialog appears. (b)
  4. Enter a prompt and change the property values. (c)
  5. Click the [OK] button. (d)
  6. The generated image appears in the [Generated Image] sub-panel.

Note: The generated image will be saved in <addon_folder>/_data/image/genereated.

Properties
Size The size of the images to generate.
Num The number of images to generate.
Name Name for a generated image.
If [Auto] is checked, the name will be generated automatically.

Edit Images

1. From Sidebar Panel

Edit Images from Sidebar Panel

  1. Go to Image Editor.
  2. Open [OpenAI] tab on Sidebar.
  3. Open a [Edit Image] sub-panel.
  4. Change the properties.
  5. Select a base image to be editted. Image need to have 'RGBA' color mode. (a)
  6. Select a mask image to specify the area to edit. Image need to have 'RGBA' color mode. (b)
  7. Enter a prompt to describe the edit. (c)
  8. Click the [▶] button. (d)
  9. The generated image appears in the [Generated Images] sub-panel. (e)

Note: The generated image will be saved in <addon_folder>/_data/image/genereated with file name edit-<Base Image Name>.png.

Properties
Size The size of the images to generate.
Num The number of images to generate.

2. From Tool

Edit Images from Tool

  1. Go to Image Editor.
  2. Select the tool [OpenAI Image Tool]. (a)
  3. Press the 'SPACE' key in the Image Editor, and the pop-up dialog appears. (b)
  4. Change the properties.
  5. Select a base image to be editted. Image need to have 'RGBA' color mode. (c)
  6. Select a mask image to specify the area to edit. Image need to have 'RGBA' color mode. (d)
  7. Enter a prompt. (e)
  8. Click the [OK] button. (f)
  9. The generated image appears in the [Generated Image] sub-panel.

Note: The generated image will be saved in <addon_folder>/_data/image/genereated with file name edit-<Base Image Name>.png.

Properties
Size The size of the images to generate.
Num The number of images to generate.

Generate Variation Images

Generate Variation Images from Sidebar Panel

  1. Go to Image Editor.
  2. Open [OpenAI] tab on Sidebar.
  3. Open a [Generate Variation Image] sub-panel.
  4. Change the properties.
  5. Select a base image used for generating the variation image. Image need to have 'RGBA' color mode. (a)
  6. Click the [▶] button. (b)
  7. The generated image appears in the [Generated Images] sub-panel. (c)

Note: The generated image will be saved in <addon_folder>/_data/image/genereated with file name variation-<Base Image Name>.png.

Audio Tool

Audio Tool can transcribe an audio data.
Audio Tool can be accessed from [OpenAI] tab on the Sidebar Panel in Sequence Editor/Text Editor or from the tool [OpenAI Audio Tool] in Sequence Editor.

Transcribe Sound Strip

There are 2 ways to transcribe a sound strip.

1. From Sidebar Panel

Transcribe Sound Strip from Sidebar Panel

  1. Go to the Sequence Editor.
  2. Open [OpenAI] tab on Sidebar.
  3. Open a [Transcribe Sound Strip] sub-panel.
  4. Change the properties.
  5. Select the sound strip to be transcribed. If you check [Selected Sound Strip], an active sound strip is transcribed. (a)
  6. Click the [▶] button. (b)
  7. The text strip with the transcription text is created on the channel.
Properties
Prompt Optional text to specify the style.
Language Language of the input audio.
Temperature Higher value makes the output more random.
Lower value makes the output more deterministic.
Result Sequence channel where the transcription result to be created.
If [Auto] is checked, the channel is selected automatically.

2. From Tool

Transcribe Sound Strip from Tool

  1. Go to the Sequence Editor.
  2. Select the tool [OpenAI Audio Tool]. (a)
  3. Select the sound strip to be transcribed. (b)
  4. Press the 'SPACE' key in the Sequence Editor, and the pop-up dialog appears. (c)
  5. Change the properties.
  6. Click the [OK] button. (d)
  7. The text strip with the transcription text is created on the channel.
Properties
Prompt Optional text to specify the style.
Language Language of the input audio.
Temperature Higher value makes the output more random.
Lower value makes the output more deterministic.
Result Sequence channel where the transcription result to be created.
If [Auto] is checked, the channel is selected automatically.

Transcribe Audio Data
Editor Text Editor

Audio Tool can transcribe an audio data.
There are 2 ways to transcribe an audio data.

1. From Audio File

Transcribe Audio from Audio File

  1. Go to the Text Editor.
  2. Open [OpenAI] tab on Sidebar.
  3. Open a [Transcribe Audio] sub-panel.
  4. Change the properties.
  5. Select the transcript object from [Audio File].
  6. Select an audio file to be transcribed.
  7. Click the [▶] button.
  8. The transcription text appears in the Text Editor.
Properties
Prompt Optional text to specify the style.
Language Language of the input audio.
Temperature Higher value makes the output more random.
Lower value makes the output more deterministic.
Result Name of target text block to where the transcript is saved.
If [Current Text] is checked, the transcription text appears in the current text.

2. From Sound Data Block

Transcribe Audio from Sound Data Block

  1. Go to the Text Editor.
  2. Open [OpenAI] tab on Sidebar.
  3. Open [Audio Tool] panel.
  4. Open [Transcribe Audio] sub-panel.
  5. Select [Sound Data Block].
  6. Change the properties.
  7. Select a sound data block to be transcribed. (a)
  8. Click the [▶] button. (b)
  9. The transcription text appears in the Text Editor.
Properties
Prompt Optional text to specify the style.
Language Language of the input audio.
Temperature Higher value makes the output more random.
Lower value makes the output more deterministic.
Result Name of target text block to where the transcript is saved.
If [Current Text] is checked, the transcription text appears in the current text.

Chat Tool

Chat Tool allows you to chat about Blender or ask about operators/properties.

Chat
Editor 3D Viewport

Chat Tool can be accessed from [OpenAI] tab on the Sidebar or from the tool [OpenAI Chat Tool] in 3D Viewport.
There are 2 ways to chat.

1. From Sidebar Panel

Chat from Sidebar Panel

  1. Go to the 3D Viewport.
  2. Open the [OpenAI] tab on sidebar.
  3. Open [Chat Tool] panel.
  4. Select [Topic] (if [New Topic] is not checked) or input a new topic name (if [New Topic] is checked). If you choose an existing topic, you can continue chatting by using old conversations. (a)
  5. Open the [Prompt] sub-panel.
  6. Input a prompt and conditions (optional). (b)
  7. Click the [▶] button. (c)
  8. The response appears in the [Log] sub-panel. (d)

You can copy a chat log to the clipboard or to the text block. (e)
If the response contains Python code, you can execute it. (f)
You can also check the error message if the code fails to execute.

2. From Tool

Chat from Tool

  1. Go to the 3D Viewport.
  2. Select the tool [OpenAI Chat Tool]. (a)
  3. Set [Number of Conditions] in the tool setting if you want to add additional conditions (optional).
  4. Press the 'SPACE' key in the 3D Viewport, and the pop-up dialog appears. (b)
  5. Select [Topic] (if [New Topic] is not checked) or input a new topic name (if [New Topic] is checked). If you choose an existing topic, you can continue to chat by using old conversations. (c)
  6. Enter a prompt and conditions (optional). (d)
  7. Click the [OK] button. (e)
  8. The response appears in the [Log] sub-panel.

You can copy a chat log to the clipboard or to text.
If the response contains Python code, you can execute it directly.
You can also check the error message if the code fails to execute.

Ask Blender Operators/Properties
Editor All

Chat Tool supports asking Blender operators/properties from right-click menu.

Ask Operator

  1. Move the mouse cursor to menu/panel items.
  2. Right-click and the menu appears. (a)
  3. When the mouse cursor is over the operator/property, you can execute [Ask]. (b)
  4. When executed, the new topic is automatically started.
  5. The response appears in the [Log] sub-panel. (c)

Code Tool

Code Tool allows you to generate a Python code with Blender Python API.

Generate Code
Editor Text Editor

Code tool can generate a Python code and display in the Text Editor.

Generate Code

  1. Go to the Text Editor.
  2. Open the [OpenAI] tab on sidebar.
  3. Open [Code Tool] panel.
  4. Open [Genereate Code] sub-panel.
  5. Input a prompt and conditions (optional). (a)
  6. Click the [▶] button. (b)
  7. Generated code can be listed on the text block.

You can execute the generated code or copy it to the text block from [Genereated Code] sub-panel. (c)(d)
You can also check the error message if the code fails to execute.

Edit Code
Editor Text Editor

Code tool can edit a Python code on the text block.

Edit Code

  1. Go to the Text Editor
  2. Open the [OpenAI] tab on sidebar.
  3. Open [Code Tool] panel.
  4. Open [Edit Code] sub-panel.
  5. Select the text block whose code to be editted. (a)
  6. Input a prompt and conditions (optional). (b)
  7. Click the [▶] button. (c)
  8. Generated code can be displayed to a new text block with the name edit-<original_text_block>.

You can execute the generated code or copy it to the text block from [Genereated Code] sub-panel. (d)(e)
You can also check the error message if the code fails to execute.

Generate Code Example
Editor All

Code Tool supports generating the code example that uses Blender Python API of operators/properties from right-click menu.

Genereate Code Example

  1. Move the mouse cursor to menu/panel items.
  2. Right-click and the menu appears.
  3. When the mouse cursor is over the operator/property, you can execute [Generate Code Example]. (b)
  4. When executed, the generated code can be displayed to a new text block with the name [Example] <python_api>.

You can execute the generated code or copy it to the text block from [Genereated Code] sub-panel.
You can also check the error message if the code fails to execute.

Execute Code Immediately

Code Tool can execute the genereated code from the prompt or audio input.
There are 4 ways to do this.

1. Execute Code from Sidebar Panel (Prompt)
Editor 3D Viewport

Execute Code from Sidebar Panel (Prompt)

  1. Go to the 3D Viewport.
  2. Open the [OpenAI] tab on sidebar.
  3. Open [Code Tool] panel.
  4. Open [Prompt] sub-panel.
  5. Input a prompt and conditions (optional). (a)
  6. Click the [▶] button. (b)
  7. Generated code is executed. (c)

You can execute the generated code or copy it to the text block from [Genereated Code] sub-panel. (d)(e)
You can also check the error message if the code fails to execute.

2. Execute Code from Sidebar Panel (Audio Input)
Editor 3D Viewport

Execute Code from Sidebar Panel (Audio Input)

  1. Go to the 3D Viewport.
  2. Open the [OpenAI] tab on sidebar.
  3. Open [Code Tool] panel.
  4. Input a conditions (optional).
  5. Click the [●] button. (a)
  6. Speak to the microphone what code you want to execute. Press 'ESC' key to cancel. (b)
  7. Generated code is executed. (c)

You can execute the generated code or copy it to the text block from [Genereated Code] sub-panel.
You can also check the error message if the code fails to execute.

3. Execute Code from Tool (Prompt)
Editor 3D Viewport

Execute Code from Tool (Prompt)

  1. Go to the 3D Viewport.
  2. Select the tool [OpenAI Code Tool]. (a)
  3. Set [Number of Conditions] in the tool setting if you want to add additional conditions (optional).
  4. Press the 'SPACE' key in the 3D Viewport, and the pop-up dialog appears. (b)
  5. Enter a prompt and conditions (optional). (e)
  6. Click the [OK] button. (d)
  7. Generated code is executed.

You can execute the generated code or copy it to the text block from [Genereated Code] sub-panel on [Code Tool] panel.
You can also check the error message if the code fails to execute.

4. Execute Code from Tool (Audio Input)
Editor 3D Viewport

Execute Code from Tool (Audio Input)

  1. Go to the 3D Viewport.
  2. Select the tool [OpenAI Code Tool]. (a)
  3. Set [Number of Conditions] in the tool setting if you want to add additional conditions (optional).
  4. Press 'Shift' + 'A' key in the 3D Viewport. (b)
  5. Speak to the microphone what code you want to execute. Press 'ESC' key to cancel. (c)
  6. Generated code is executed. (d)

You can execute the generated code or copy it to the text block from [Genereated Code] sub-panel on [Code Tool] panel.
You can also check the error message if the code fails to execute.