Dave: What are the practical applications of the project we’ve recently completed for our US client who publish science content?
Justin: Many businesses are grappling with understanding how to effectively utilise generative AI and RAG frameworks in content delivery. For this particular client, the aim was to drive publication sales by providing accurate lists and summaries of available content. Initially, they tested generative AI and augmented search internally and with partner organisations to ensure accuracy and usability. The primary concern has been ensuring their AI-generated summaries are precise and not misleading, especially in educational contexts like research papers. The focus lies in creating prototypes for conversational search, either for all or a subset of content, to validate usability before full implementation. This journey involves verifying that the generated results are accurate and reliable, without conflating concepts, and ensuring they meet user expectations. The current phase involves internal testing and collaboration to refine the system before wider deployment.
Dave: Sounds expensive?
Justin: Many assume it's costly to reach this stage, so few companies are exploring it. However, we’ve developed a cost-effective, repeatable framework, ensuring initial prototyping can be affordable before scaling up production.
Dave: What we’re doing isn’t unique, but it does require skill and experience.
Justin: We’re using existing technology. The crucial skill lies in understanding client data, user preferences and assembling the appropriate components into a cohesive system. Opting for locally hosted language models ensures privacy for prompt data and fine-tuning information. The value lies in comprehending the technology and its myriad options, selecting the most suitable components, and educating clients on effective usage, including its limitations.
Dave: Once implemented well, it can be evolved though. It's an emerging technology - things are moving fast. So what we’ve created for this client is scalable - and changeable and adaptable?
Justin: Definitely. Our clients are always surprised by the diverse abilities of large language models. Trained primarily for classification and text completion, they excel in unexpected areas like assessment tasks. Moreover, their agent/toolset capabilities enable them to to be integrated with a wide range of data sources and tools, enabling complex tasks to be accomplished using natural language prompts.
Dave: How has the user journey changed with the client’s new facility, and how does it enhance the process?
Justin: With conversational search, users can now ask the search engine natural questions. Instead of relying solely on specific keywords, and receiving a ranked list of results, they get contextually and semantically relevant content summaries. We employ a mix of vector-based and keyword-based searches, feeding the results into a large language model for summarization.
Dave: You’ve told me this before, and be gentle with me… what do you mean by vector-based search?
Justin: Vector-based search generates embeddings for content pieces, storing them as vectors akin to spatial representations. When users search, their queries are also transformed into vectors, facilitating distance calculations between similar items. Unlike traditional keyword-based searches, which focus solely on textual appearances, vector-based search prioritises proximity relationships among vectorized texts, offering contextually similar results. Users receive search result lists summarising relevant information or addressing specific queries, enabling quick access to potential answers without delving into supporting documents. This approach streamlines information retrieval, ensuring users encounter more pertinent results compared to keyword-based searches, which often yield excessive or irrelevant outcomes.
Dave: When you talk about vector-based search generating ‘embeddings…’
Justin: Vector embeddings represent data and its context numerically, stored as dense vectors in high dimensions. These embeddings are produced by models trained on vast datasets, enabling them to provide highly relevant and precise outcomes.
Dave: So where's it going? What's the next phase?
Justin: We’ll add interactivity to large language models and personalised vectorized content using agents. These agents act as intermediaries controlled by the language model based on user input. For instance, a maths agent can perform simple calculations on request, while another can retrieve specific information from websites like Stack Overflow, summarising relevant results. Future developments aim to integrate graphical and statistical representations into conversational searches, allowing users to explore data by asking questions instead of navigating through filters manually.
It goes on and on. For this particular client, for example, instead of navigating to a website to make a purchase, you could simply ask to subscribe to the top 3 papers. A payment form would then appear within your UI, keeping you within the conversational context. With advancements in speech-to-text and text-to-speech, interacting verbally rather than typing will be standard stuff within a year. The accuracy will be sufficient to engage interactive agents seamlessly, performing tasks related to the ongoing conversation theme. Tasks like sending items, adding to bookmarks, or translations, will be executed swiftly and seamlessly, saving time and enhancing conversational efficiency.
Dave: What we’re doing for one of our market research clients builds on this, it seems?
Justin: Theirs involves developing interactive agents and training a localised version of the large language model on their specific content, which features domain-specific language within marketing insights and trends. The work is distinct, but we've reused and adapted our UI and conversational search infrastructures over time, ensuring reusability and quick deployment for ongoing clients. The customizable aspect lies in understanding and integrating their data, as well as fine-tuning the language model prompts as needed. This standardised approach to productizing our work allows for efficient iterations and enhancements to the search interface, while accommodating unique client requirements.
We’re doing similar for an educational client; creating a prototype, to grade essays and student submissions using rubrics, and which provides feedback for improvement.
Dave: Before we wrap up, we often talk about our concerns about responsibility regarding AI. Are you worried about individuals setting up and training these models without due care?
Justin: Regarding behaviours seen in certain large language models and their learning mechanisms, there can be a lack of understanding, from some who have developed and trained these models, of the reasons behind these behaviours. While these models can be astonishingly powerful, often surpassing initial expectations, there exists a fundamental gap in understanding how they actually accomplish tasks.
Dave: And because it’s not fully understood, there's no way of policing it?
Justin: That's one of the risks, yes. There are various initiatives in place. Elon Musk is suing OpenAI over perceived deviation from initial societal-benefit focused goals towards profit. In the EU, an AI task force has just released the first version of the AI Act; the result of almost 3 years of discussions, which goes some way to creating a framework within which businesses, individuals and regulatory bodies can work with AI and derivative technologies. Mustafa Suleyman warns of the dangers posed not just by AI, but also by the convergence of various technologies, such as 3D-printing biological and physical matter simultaneously.
Dave: Biosynthesis…
Justin: Yes, which means you can print DNA, for example, and therefore you can use AI to charge through huge numbers of possible DNA sequences for all sorts of different things. You can buy a DNA printer that you can run in your garage for about 40K and start creating organisms at home. The potential ramifications of combining these technologies irresponsibly is a concern, but hopefully also a real opportunity for humanity to heal and protect itself.
Originally published on LinkedIn
