What in the AI World Is RAG?

Understanding Retrieval-Augmented Generation and how it helps the enterprise build smarter AI apps.
, , ,
"The AI Economy," a newsletter exploring AI's impact on business, work, society and tech.
This is "The AI Economy," a weekly LinkedIn-first newsletter about AI's influence on business, work, society and tech and written by Ken Yeung. Sign up here.

Have you heard about Retrieval-Augmented Generation, or RAG, in the world of AI? It’s a technique used with generative AI. Even though not many people know about it, when applied, it plays an important role. While we often focus on fancy AI that generates cool pictures or helps us find answers, RAG is a big deal for making sure chatbots or AI apps give accurate and helpful responses.

For this week’s issue of “The AI Economy,” I spoke with Dennis Perpetua, the Global CTO for Digital Workplace Services and Experience at IT provider Kyndryl. He educated me on RAG and how the technique is being applied in the enterprise to build smarter applications.

Plus, we explore some of the women leaders who have shaped the AI space, thanks to a series published by TechCrunch.

Programming Note: “The AI Economy” will not be published next week as I’ll be in San Francisco attending the Llama Lounge event. Say hi if you’re there! This newsletter will return the following week.

The Prompt

“The two most important things that RAG does, relative to the enterprise, is it allows us to source the answers, and have that be traceable.”

That’s according to Perpetua, who is a Distinguished Engineer at Kyndryl and before that, at IBM. In his role, he helps organizations modernize their business processes, including by using AI.

RAG extends the capabilities of Large Language Models (LLMs) to specific domains or an organization’s internal knowledge base. It’s perhaps more commonly used in the enterprise.

Integrating an LLM with RAG is “relatively simple,” but he says it’s critical developers address how traceability is fed back into the system and that content audits are conducted. Additionally, content management must not be overlooked.

Having a large knowledge bank is not a prerequisite for using RAG as it’s subject to the use case. But for those scenarios involving highly specific company concerns, Perpetua tells me the utility of RAG increases significantly.

He explains RAG is beneficial mostly in closed systems, those in which organizations can dictate the type of information fed into the AI. By having the AI free from current events, political ramblings and other non-relevant information, it’ll have access to data centered on responding to customer-specific questions, reducing the risk of misinformation or worse.

Perpetua stated that non-experts, or those without domain expertise, will find RAG particularly beneficial as the technique becomes a safety backstop when providing customer support, product repair troubleshooting, or anything else. The response the AI provides would be traceable back to a verified source.

“Traceability is the assurance that is there to make sure that when you get the answer, you can click on it and say, ‘Alright, this is where this is pulling from.’ It turns into a very accurate and interesting Table of Contents,” he explains.

▶️ Read more about RAG in my full interview with Dennis Perpetua

A Closer Look

The New York Times was blasted last year when it published a piece exploring the researchers, tech executives and venture capitalists who worked on AI. None of them were women. The piece highlighted the roles Google co-founder Larry Page, OpenAI CEO Sam Altman, Microsoft founder Bill Gates, LinkedIn founder Reid Hoffman, Professor Geoffrey Hinton, Microsoft CEO Satya Nadella, Mark Zuckerberg, Elon Musk and investor Peter Thiel played.

What about Stanford’s Fei-Fei Li, Turing Award winner Yoshua Bengio, former Association for the Advancement of Artificial Intelligence President Barbara Grosz, ex-Googler and computer scientist Timnit Gebru, roboticist and computer scientist Lucille Davis and others like them?

TechCrunch has published a series examining contributions from some notable women in the field of AI.

Despite the many ways in which women have advanced AI tech, they make up a tiny sliver of the global AI workforce. According to a 2021 Stanford study, just 16% of tenure-track faculty focused on AI are women. In a separate study released the same year by the World Economic Forum, the co-authors find that women only hold 26% of analytics-related and AI positions.

To its credit, the publication plans to publish more profiles spotlighting women in AI to continue raising awareness over the field’s gender disparity. However, it’s worth reading the first few profiles published this week.

Check out these interviews:

Today’s Visual Snapshot

A look at how select AI company stock prices changed on Feb. 22, 2024. Image credit: Axios
A look at how select AI company stock prices changed on Feb. 22, 2024. Image credit: Axios

The AI boom helped Nvidia overtake Amazon in market capitalization, as highlighted in last week’s issue. This week, the chipmaker not only surprised analysts with its latest quarterly earnings report but saw its market cap surpass $2 trillion in intraday trading for the first time.

Fortune does not seem to be limited to just Nvidia, however. The euphoria over AI is being felt by its industry peers, with stock prices by infrastructure players rising over the past few months.

The real winners in AI aren’t the software makers, but the ones making the chips that power it all. But with demand soaring, can the companies who make the processors and high-end servers keep up?

▶️ Read this interview with Nvidia CEO Jensen Huang on the company’s success

Quote This

“The kernel of truth is we think the world is going to need a lot more (chips for) AI compute. That is going to require a global investment in a lot of stuff beyond what we are thinking of. We are not in a place where we have numbers yet.”

— OpenAI CEO Sam Altman when asked a question about whether he is trying to raise as much as $7 trillion to fund a chip venture.

Neural Nuggets

An AI-generated image of a robot reading a newspaper.
An AI-generated image of a robot reading a newspaper.

🏭 Industry Insights

🤖 Machine Learning

✏️ Generative AI

⚙️ Hardware

🔬 Science and Breakthroughs

💼 Business and Marketing

📺 Media and Entertainment

💰 Funding

⚖️ Copyright and Regulatory Issues

💥 Disruption and Misinformation

🔎 Opinions and Research

End Output

Thanks for reading. Be sure to subscribe so you don’t miss any future issues of this newsletter.

Did you miss any AI articles this week? Fret not; I’m curating the big stories in my Flipboard Magazine, “The AI Economy.”

Follow my Flipboard Magazine for all the latest AI news I curate for "The AI Economy" newsletter.
Follow my Flipboard Magazine for all the latest AI news I curate for “The AI Economy” newsletter.

Connect with me on LinkedIn and check out my blog to read more insights and thoughts on business and technology. 

Do you have a story you think would be a great fit for “The AI Economy”? Awesome! Shoot me a message – I’m all ears!

Until next time, stay curious!

Subscribe to “The AI Economy”

New issues published on Fridays, exclusively on LinkedIn

Subscribe

Leave a Reply

Discover more from Ken Yeung

Subscribe now to keep reading and get access to the full archive.

Continue reading