Building AI Products- Perspectives from CIOs, VCs, and Product Leaders

Introduction

We all know that artificial intelligence is transforming our work and our lives.

Now, to give you all some more perspective, we want to illuminate some eye-opening statistics.

According to Grand View Research’s report, the global AI market was valued at $39.9 billion in 2019 and is expected to reach $733.7 billion by 2027, growing at a CAGR of 42.2% from 2020 to 2027.

Not just that Accenture has found that AI has the potential to increase the company’s productivity by 40%

As we peel back a little and dive deeper into the opportunities and challenges posed by AI in product development, we are super excited to bring you insights from the leading experts in the field. In this blog, we will analyze the engaging fireside event hosted by Chisel on the topic “Building AI Products: Perspectives from CIOs, VCs, and Product Leaders.” This discussion dissects the practical experiences and visionary strategies of industry leaders at the forefront of AI innovation.

So, let’s stir the pot and get going!

Kickoff and Opening Remarks

Adopting artificial intelligence brings both promise and peril for enterprises. “The best things about AI haven’t been invented yet,” noted Yousuf Khan from Ridge Ventures. However, security concerns around how customer data will be used and the potential for biased outcomes have given many CIOs pause.

To discuss these issues, Chisel hosted an event bringing together panelists from different perspectives including,

• Chetna Mahajan: (Chief Digital & Information Officer at Amplitude, ex-CIO at ZoomInfo)
• Yousuf Khan: (Partner at Ridge Venture, ex-CIO at Moveworks, Automation Anywhere, and Pure Storage)
• Ashish Agarwal: (Founder, CPO & CTO at Productiv, ex-VP of Eng at Postmates, ex-Amazon and Microsoft)
• Praful Chavda: (Founder & CEO at Chisel, ex-founder at Bryght.ai, ex-Product Leader at Microsoft)

The goal was to have an open discussion on enterprises’ key concerns when adopting AI solutions and what vendors can do to address them. The event covered how the due diligence process has changed when purchasing AI software; different use cases enterprises are interested in for AI, and perspectives from a CIO, product leader, and investor.

Expert Insights and Panel Discussion on Building AI Products

As we started off the discussion, the panel experts shared their perspectives on the transformative power of artificial intelligence. Insights included market growth, productivity enhancements, and the potential of AI to revolutionize various industries.

For our readers’ ease we have divided the discussion part into 5 sub heads.

So lets get started

Seeking AI’s Most Promising Use Cases

Praful initiated the conversation with a thought-provoking question to the panelists:

“Let’s start with Yousuf. As a VC, what AI use cases or areas excite you the most for investment? Chetna, as a CIO, what AI scenarios are you looking to invest in for your organization? And Ashish, from both a product leader and a SaaS management perspective, where do you see the most interesting AI investments happening?”

Yousuf’s Perspective: The VC’s Enthusiasm for AI

Yousuf highlighted the surge of interest in AI post-November 2022, spurred by the rapid adoption of tools like ChatGPT. This widespread excitement has created numerous opportunities for developing AI-driven products. 

Yousuf emphasized the following points:

• Vertical Use Cases: Teams identify specific industry problems and create AI solutions, such as accelerating workers’ comp insurance claims or enhancing computer vision on factory floors.
• Internal Business Processes: AI applications that streamline and optimize internal operations, like demand forecasting and improving sales outcomes.
• Revenue Impact: Solutions that leverage AI to enhance sales and revenue, such as using data to drive actionable insights.

Yousuf noted that the best AI innovations are yet to come, highlighting the potential for continued growth and discovery in this field.

Chetna’s Insights: The CIO’s Focus on AI for Productivity and Process Improvement

Chetna provided a mature perspective on AI investments, detailing several vital areas where AI is making a significant impact at Amplitude:

• AI Assists Tools like copilot for code generation and AI for customer service aimed at boosting productivity.
• Content Generation: AI applications for marketing, sales, and HR functions, including auto-generating emails and job requisitions.
• Data Quality and Lead Management: Improving the quality of leads and enhancing sales productivity through AI-driven insights.
• Enterprise Search: Implementing tools like Glean to enhance information retrieval and accessibility across the organization.
• Process Mining and AI Avatars: Exploring AI-driven process optimization and using AI avatars for internal enablement and training.

Chetna expressed curiosity about AI solutions that directly impact the top line, acknowledging the evolving nature of AI’s role in business.

Ashish’s Take: The Product Leader’s View on AI Integration

Ashish emphasized the experimental nature of AI integration in the current landscape, noting two significant trends

• Curiosity and Experimentation: Organizations are keen to explore AI, with widespread use of tools like ChatGPT for various tasks, despite its unofficial status as a “Shadow IT” app.

• Enhancing Existing Products: Many companies are incorporating AI features into their existing offerings, such as AI-driven meeting summaries in Zoom or GitHub Copilot for code assistance.

Ashish highlighted the challenge of creating impactful AI products, stressing the importance of relevance and user satisfaction. He underscored the need for new frameworks and strategies to develop and deploy AI solutions effectively.

The discussion revealed a common theme: AI’s transformative potential is immense, but harnessing it requires a nuanced approach.

Some of the Key takeaways from the initial question were-

• Focused Applications: Identifying specific problems and tailoring AI solutions can drive significant value.
• Productivity and Process Optimization: AI assists, and process mining substantially improves efficiency and streamlining operations.
• Continuous Exploration: AI transforms, necessitating experimentation and adaptation to discover the most effective applications.
• User-Centric Design: Ensuring user-friendly and relevant AI solutions is crucial for successful adoption and long-term impact.

As AI continues to advance, the ability to strategically invest in and implement AI technologies will be a defining factor for businesses aiming to stay ahead in an increasingly competitive market.

Key Concerns From Enterprise Buyers

One of the primary focuses of the discussion was on the key concerns enterprises have when adopting AI solutions. Chetna provided insight from her perspective as a CIO, noting that data and model security are at the top of her mind. There is uncertainty around how customer information will be protected and used, both by the vendor solution itself and by any underlying AI platforms or infrastructure.

Bias and ensuring ethical outcomes is another primary consideration.

Yousuf explained that solutions like AI-based sales forecasting could lose all credibility if biased results are produced. With recruiting tools, bias could seriously damage a company’s reputation if not adequately addressed. Vendors must demonstrate how they evaluate and mitigate bias in their solutions.

Continuing to support and update models over time was also identified as an expectation from enterprise buyers. Ashish discussed how more than mere assurance from a vendor is needed – customers want transparency into how models are secured and will remain up-to-date as needs and technology evolves. Given the dynamic nature of AI models, this is a new challenge compared to traditional software.

Finally, Chetna emphasized the importance of enterprises genuinely understanding how AI products work under the hood. Buyers need visibility into the decision-making process behind recommendations or classifications. This helps determine functionality and how to avoid potential issues like bias. Some newer companies are more open about exposing their model architectures than others.

Advice From Vendors

When addressing buyer concerns, the panelists provided several recommendations for vendors. Transparency was a key theme—Yousuf emphasized the importance of clearly communicating exactly how customer data will and will not be used in model training and ongoing security and management. Technical experts also need to be actively engaged during sales discussions. As Ashish noted, buyers want to avoid boilerplate responses but genuinely want to understand product functionality from those who can speak in depth to the technical details and stand behind the solution.

Product positioning was another area of focus. The AI landscape remains complex with no dominant solution, so vendors must clearly show why their offering stands out. Yousuf advised creating a formal “buyer’s guide” to lay out how customers should evaluate different solutions based on hard evidence rather than just assurance. Finally, building trust requires going beyond documentation to demonstrate processes in action. Process auditing and allowing customers witnessing data deletion or anonymization firsthand help substantiate vendor claims, as Ashish’s example of screen sharing a data removal illustrated.

The message to vendors was that transparency, expertise and substantiation are vital to addressing enterprise concerns in a dynamic, complex AI environment. Customers expect authentic dialogue and proof points, not just marketing speak, to feel comfortable adopting emerging technologies with their organizations.

Panelist Perspectives

As the conversation progressed, the Panelists dabbled into their experiences. Now, let’s dig deeper into their perspectives.

Beyond providing overarching advice, each Panelist offered valuable insights based on their unique roles and experiences. Chetna discussed use cases Amplitude explores, like process mining and AI avatars to enhance training. As a CIO, she emphasized challenges such as enabling the field on new technology rollouts.

From his vendor viewpoint, Ashish highlighted common scenarios Productiv sees, such as customers building their own AI solutions or using assistants. He described exciting conversations that got into the technical “weeds” of ensuring data boundaries.

As an investor and ex-CIO, Yousuf drew on various industry examples. He described demand forecasting and recruiting as appealing AI applications. The Formula 1 sensor anecdote also illustrated the importance of not just data but human judgment.

Praful provided context on Chisel’s mission to integrate feedback and automation. He noted customers’ disparate reactions to AI, from excitement to caution. The audit is pending. Some valued being able to select Chisel’s underlying AI provider depending on internal approvals to further build trust.

Through such war stories and use cases across panelist roles, the discussion offered valuable comparison points and tangible takeaways for the AI adoption journey.

Praful, The CEO of Chisel, helped ensure a free-flowing yet informative dialogue between experts and the audience.

Examples Discussed

As the dialogue advanced, the panel thoroughly probed into some examples of adopting AI responsibly. One example that Yousuf brought up centered around security questionnaires and how some companies were automating the process of reviewing and providing feedback on questionnaires in a concerning way. He described how sales teams could get responses back extremely quickly, which raised red flags. Upon investigation, it was uncovered that the automated system was simply redlining submitted questionnaires and sending them back without a thorough review. This highlighted the importance of carefully evaluating how AI systems are used and the potential for unintended consequences if not developed responsibly.

Praful then echoed this point about understanding the implications and “blast radius” of getting AI implementations wrong. He talked about how minor errors with other types of software may not cause major issues. Still, with AI, there is a higher risk given its ability to make decisions and take action autonomously. Yousuf furthered this discussion by sharing an anecdote about Formula 1 racing. He told a story about a driver who correctly identified an unusual loss of power, which was later traced to a magnetic field from a nearby bridge that the sensors had not picked up on. This real-world example demonstrated the value of human intuition and judgement when combined with data-driven insights.

The panel discussion meaningfully explored challenges around responsible AI adoption through relevant examples and cautionary tales, emphasizing the importance of vigilance, oversight and accounting for unknown unknowns. This built upon Praful’s key point about understanding the broader implications and higher stakes involved with AI systems.

Audience Q&A Session

Several thought-provoking questions were raised as the amazing conversations moved to an audience Q&A session. One attendee inquired about who product companies typically send to address concerns over complex AI topics. Ashish stressed the importance of depth in these situations rather than surface-level responses. He recommended pairing a senior leader who can stand behind claims made with a technical expert who can discuss implementation details.

Another question focused on whether scrutiny of AI adoption differed depending on whether the technology was core to a product or more ancillary. Chetna shared Amplitude’s perspective of applying diligence but varying the intensity based on existing trust in a vendor. For established platforms adding new features, vetting is still required, but less so than for entirely new solutions.

Karthik then posed a thoughtful query about gaining assurances when products rely on third parties like OpenAI rather than internal models. This opened discussion around contracts, auditing and transparency. While no solution is perfect, the panel emphasized establishing trust through open communication of capabilities and limitations rather than vague promises. Buyers ultimately seek confidence that controls are in place to ensure responsible use of data and ongoing model oversight.

Jugal asks about the preference for single-tenant versus multi-tenant deployments when purchasing products. Chetna prefers multi-tenant deployments, citing better control and failover capabilities. Yousuf concurs, emphasizing that multi-tenant deployments typically offer more redundancy and better scale. While some regulatory environments might necessitate single-tenant solutions, multi-tenant options are generally preferred unless specific exceptions are required.

The Q&A continued the enlightening back-and-forth around challenges of building, evaluating and purchasing AI-powered offerings. By engaging thoughtfully with audience perspectives, the panel shed valuable light on considerations from both vendor and enterprise points of view. Their fruitful exchange emphasized collaboration over simple answers in navigating this complex but important domain.

Closing Thoughts and Future Outlook

And, just like that, we arrived at the finish line of the event.

The speakers offered their closing thoughts and a future outlook on the continued evolution of AI and its impact on product development.

As the discussion drew to a close, several themes emerged as key takeaways. The panel reiterated enterprises’ top concerns around AI security, bias mitigation, and model maintenance. However, opportunities also exist when these challenges are adequately addressed. Both vendors and customers must play active roles. While vendors need transparency to alleviate worries, buyers would be remiss not to consider AI’s benefits with oversight.

Overall, the event highlighted dialogue and substantiation as paramount on the enterprise AI journey. By exchanging perspectives and anecdotes, panelists and audience alike gained a more profound understanding of changing due diligence processes and the diverse ways AI impacts industries. For participants and observers, the panel format proved a stimulating means to cover complex topics with nuanced back-and-forth.

Praful concluded by thanking the experts for their time and perspectives. Their insightful sharing of challenges and lessons learned will continue to benefit others who are navigating the AI space. Events like these help foster the kind of understanding and community that can propel responsible innovation. For all attendees, from CIOs to investors, the discussion underscored both the work still ahead and the collaborative opportunities that lie in store for AI’s continued development.