Option to enable chat streaming #1516
Conversation
TamiTakamiya
force-pushed
the
TamiTakamiya/AAP-39043/option-to-enable-chat-streaming
branch
from
3cfc3c8 to
aadf74e
Compare
|
@TamiTakamiya I'll look again tomorrow. If it's urgent perhaps ask someone else to look. |
|
@manstis I have pushed the changes based on your comments. A new environment variable is no longer included. Please take a look. Thank you. Notes:
|
|
@manstis It's not urgent. Pls take time :-) |
ansible_ai_connect/main/views.py
Outdated
| @@ -140,6 +140,11 @@ def get_context_data(self, **kwargs): | |||
| context["user_name"] = user.username | |||
| context["debug"] = "true" if settings.CHATBOT_DEBUG_UI else "false" | |||
|
|
|||
| llm: ModelPipelineChatBot = apps.get_app_config("ai").get_model_pipeline( | |||
There was a problem hiding this comment.
Hint for optimisation (not that what you have written is slow/expensive to call).
You could have got the instance in a __init__ function...
... and used this in L#146 and L#126-L#130.
There was a problem hiding this comment.
@manstis I have update the code based on your comment. Pls re-review this when you have time. Thanks!
There was a problem hiding this comment.
@TamiTakamiya LGTM 👍 but I am confused as to whether stream is a static configuration parameter or a dynamic runtime variable.. Please see my comments.
|
TamiTakamiya
deleted the
TamiTakamiya/AAP-39043/option-to-enable-chat-streaming
branch
Jira Issue: https://issues.redhat.com/browse/AAP-39043
Description
Add an option to enable chat streaming to our AI Connect service Django application. The new option is
and its default value is set to
False. UnlessCHATBOT_STREAMis set toTrue, the chat streaming support code willnot be enabled.
Testing
Steps to test
Scenarios tested
Unit tests are provided with this PR.
Production deployment