How can one replace the UI of an application with an LLM's chat window? The bot should be able to do everything it used to but via natural language. So the end user doesn't have to click at buttons or view options in a menu; rather, he/she should be able to tell this via simple sentences, which can trigger the usual APIs that were event (click/hover) driven. Are there any existing projects in github or a definite approach to solving this?
Asked
Active
Viewed 34 times
1
-
Do you think, that writing would be more convenient than clicking buttons? – Hritik Sharma Jun 13 '23 at 12:03
-
Yes, definitely. For complex UIs having a lot of options, menus, drop-downs, etc. it's best if the user tells what's needed. An agent which knows what's in the UI can do the task accordingly by asking details that the user might have missed while prompting. – theodre7 Jun 13 '23 at 12:49
-
But then, user will have to write a long form of what he/ she wants, which is not preferable. Clicking a button or selecting is a better in terms of ui/ ux. Though, it is good to experiment on this idea. – Hritik Sharma Jun 13 '23 at 13:01
-
Yes. That's a viable pitfall. But consider this. The user will give a small prompt, which the LLM will use to understand the intent. After that, it'll converge on what the user actually wants by asking more questions – theodre7 Jun 15 '23 at 05:56