my_ml_notes

Generative AI and Business Needs

This section show my notes on business considerations while talking about generative AI.

Cost of Generative

To better understand the cost of generative AI usecases, we describe in this section the main elements to consider in order to calculate the cost of generative AI applications.AI

Picture below helps in understanding the multifaceted nature of expenses involved in deploying and maintaining generative AI solutions.

Here are some of the cost related to generative AI, which are composed by:

• Application Implementation:
  • Custom Application Build: Costs associated with building or modifying the custom application tailored for generative AI.
• Model Cost:
  • Foundation Models: Costs related to using base or customized foundational models for AI.
• Infrastructure Cost:
  • Cloud Infrastructure: Expenses related to cloud infrastructure required to support AI operations (either through an API or by deploying those models on computing instances).
• Data Cost:
  • Enterprise Data: Costs for acquiring and managing enterprise-specific data.
  • Data Science & Engineering: Costs for data science and engineering efforts.
  • Data Governance & Management: Expenses for managing and governing data.
  • Public/Vendor Data: Costs associated with obtaining data from public sources or vendors.
• User Cost:
  • User License: Costs for licensing users.(e.g. code assistant)
  • Enablement/Training: Expenses for training and enabling users to effectively use the AI system.
• Consumption / Inference Cost:
  • Context Size: prompt+answer: Costs depending on the size of the prompt and the answer generated.
  • Number of Interactions: Costs based on the number of interactions.
  • $ per token / computing instance: Costs calculated per token or per computing instance used.

Generative AI Usecases Categories and KPIs

The amount of investiment in generative AI lately has skyrocket. It jumpt from 2.85 billion US dollars in 2022 to 25.23 billion US dollars in 2023. (ref. IEEE Spectrum).

This has also been reflected on the race on the foundation models lately, that is clearly seen by number of foundation models been released by big industry players like Amazon, Google, Meta, Microsoft and OpenAI for example.

Generative AI is impacting every organization nowadays and an important aspect is how those organization can access the ROI (return of investiment) for generative AI. As described by Gartner, generative AI usecases can be divided in the following mix when considering value versus risk:

Quick wins: task specific usecases with productivity assistants such as Amazon Q for Developer and Microsoft 365 Copilot. Integrating these capabilities into other business process can help you maintain a competitive edge. Examples are: assistants/copilots, marketing generation and code assistants.

Differentiating use cases: industry specific or custom application that leverage your enterprise data in new ways extending your current process. Application such as Retrieval Augumented Generation (RAG) enables you to leaverage your enterprise data through for example the use of Amazon Q for Business or Amazon Bedrock. This might lead to potential benefits such as revenue generation. Additional examples are: custom support apps, sales apps, and enterprise document search and summarization.

Transformational initiatives: has the potential to turn business and model upside down. Those transformative usecases comes with complexity, risk and cost and high potential for technical depth. Examples are the creation of fine tuning foundation models or even custom pre-trained foundation models such as BoombergGPT - a foundation model specialized in financial domain.

KPIs examples for Generative AI Applications

Implementing and closely monitoring Key Performance Indicators (KPIs) is crucial to ensure that the costs associated with generative AI projects are managed effectively and remain within budgetary constraints.

Example of KPIs per usecase mix that might be relevant to keep track on are:

• Quick wins
  • number of stories completed, commits, changes per week
  • deployment frequency, pull request interation time
  • lines of code per developer, dev retention
  • time to upskills associates/junior developers
• Differentiating use cases
  • time saved to perform tasks (e.g. answer customer support question, create document/process/marketing)
  • employee retention
  • portion of time spent on customer / doing knoledge tasks
  • Time to create content / analyze data / derive insights
  • rework time (i.e. output quality)
  • content output throughput (amount of task per resource)
  • call sastisfaction scores
  • Time to expert skills levels
  • Amound of escalations
  • incremental revenue on upsell/cross sell.
  • customer retention / customer satisfaction
  • Content quality / error rates
• Transformational initiatives
  • market share
  • generative AI product revenue.
  • amount of market created
  • time saved to access risk and complexity of contracts/insurance documents
  • Claim cost/processing improvement
  • Brand improvement
  • reduce time to market (e.g. new drug discovery)
  • reduce error rates (e.g. in medicine, law and finance)
  • increase on opportunity and risk prediction

References:

Generative AI Risks

Some News Examples:

Driver uses ChatGPT hack to get dealer to agree to sell new car for $1 in ‘legally binding deal’ in blow for AI rollout - https://www.thesun.co.uk/motors/25091054/driver-uses-ai-loophole-buy-new-car-1/

Source: https://abovethelaw.com/2024/01/keep-your-firm-far-away-from-whatever-ai-chevy-was-using/

This site is open source. Improve this page.