Problems Product Managers Face When Building Products With Large Language Models (LLMs): And How To Counter

Conversational AI and text UIs are having a moment. But there are headaches in building with the tech. Context is lost over time with long conversations. Chaining is hopeful but not there yet. The current level of latency is unacceptable for some real-time interaction patterns.

Emerging techniques to combat these shortcomings include chain of thought (CoT), few-shot prompting, and think step-by-step.

Shoehorning LLMs into existing products will be hit or miss. The same thing happened with the transition from desktop to mobile. There it became important to play to the platform’s strengths. For mobile, it was the camera, gestures, location, and offline storage. The same approach should be taken for LLMs. Don’t try to take traditional web form design patterns and shoehorn chat UIs into the product.

The key to remember is that LLMs don’t need to be tied to a traditional chat UI. It’s not about forcing users to text-messaging interface. Here are three examples where an app can play to the strength of an LLM.

  1. Focus on business documents and coding scenarios where long-form writing already occurs. This makes it more natural to integrate.
  2. Remove the human interaction altogether and have your code plus the LLM complete work on their behalf (sending a notification when the work is done).
  3. Build in hyper-niche segments where models can be fine-tuned to a high degree of accuracy.

Play to the strength of LLMs, and the immediate opportunities are obvious.

Related Posts

6 Steps To Write A Chat Plugin For ChatGPT & Microsoft Platforms — This is low dev effort and high value — if your API has valuable business logic and data
Chat Plugins: This Seemingly Simple Tech Is Going To Spawn A Whole New Vintage Of Successful Businesses — The tech wasn’t readyThe business model was weakMy pitch didn’t get fundedBooBut succes
20 Chat Plugin Use Cases That Will Be Table Stakes In 18 Months — Here are 20 use cases that will become table stakes within 18 months🙋‍♂️ Answer employee q

Get money. Keep money.

Join 2,000+ subscribers to my free weekly newsletter — Infinite Runway. Every Sunday morning, you'll get one actionable tip to help start and automate your business.