Using LangChain, programmers have been able to combine ultrasound imaging for applications such as breast cancer diagnosis with a ChatGPT-like natural language prompt. Korea Institute of Advanced Science and Technology
Certainly, generative AI, through ChatGPT, is perhaps best known for its cooking recipes and automated code programming. But a new open-source framework based on large language models gives a more practical, concrete, but above all professional orientation to GenAI.
LangChain, which has been around for a little over a year, is what one might consider a gentle introduction to programming artificial intelligence agents. How ? Through a very simple set of libraries based on GenAI models. The open source technology is backed by a startup of the same name, which offers a server platform for commercial deployment of applications built with LangChain.
The essential, and perhaps distinctive, point of LangChain is that it combines a large language model with various external resources.
Why a GenAI framework?
So it can retrieve data from a database, for example, and pass the results of the language model to an application, get the results from that application and send them back to the language model.
The framework therefore makes it possible to chain resources, each resource becoming in a way an agent which deals with part of the problem in the context of the language model and the prompt.
And there are already fascinating examples of practical uses in different disciplines of this technology.
Natural language and breast cancer diagnosis
For example, using LangChain, programmers were able to combine ultrasound imaging for tasks such as breast cancer diagnosis with a ChatGPT-like natural language prompt. A radiologist can therefore call upon the computer as an analytical assistant with a sentence such as “Please give me a summary of the observations given based on the position of the probe” in a breast ultrasound image.
An interesting aspect of the program, designed by Jaeyoung Huh and colleagues at the Korea Advanced Institute of Science and Technology, is that it brings together three different neural networks of a widely used variety — ResNet-50 — which is a neural network of vision recognition, and which therefore excels in image classification.
Each of the three neural networks is trained separately to perform three tasks:
- Identifying a suspicious shape in an ultrasound image
- The classification of this form
- Identifying the location of the shape in the body
LangChain’s goal is to integrate these three networks into natural language commands, such as “give me probe information for the given image” and then “give me a summary of the given observations.”
At the simplest level, LangChain can therefore be a way to create a user-friendly interface for AI, of the type that has long been dreamed of by specialists in fields such as medical AI, who are looking to create a medical assistant responding to voice commands.
An AI response reliability scoring system with generative AI
One of the other objectives of LangChain is to try to eliminate the famous hallucinations of GenAI, that is to say the propensity of programs to assert false information. How ? By basing technology on authoritative external sources.
Accenture thus describes a system allowing financial predictions to be made via a “financial chatbot”.
The system takes cells from a spreadsheet and converts them into natural language statements about the data, which can then be searched to find a phrase that matches a question.
Here’s how it works:
- A user asks the language model a natural language question such as “How are my sales doing?”
- This question is fed into a model which generates a more precise question for the language model
- This prompt can include more words from the detailed question than a person would think of, in order to produce a better prompt.
- The enhanced prompt triggers a keyword search, and that search determines which of the phrases – built from tabular data – refer to the most relevant data in the table (e.g. sales, profits).
- Once the relevant data is retrieved, a second set of models helps the chatbot formulate a response to the query using the phrases from the tabular data in a chat response.
Accenture
Accenture has failed to eliminate hallucinations. Instead, Accenture designed a “confidence” rating mechanism whereby the chatbot checks its answers against the question, sees how well they match, and then assigns a high, medium, or low confidence rating to its answer.
“The confidence score indicates to the user that they should exercise caution when making key decisions using medium or low confidence responses,” Accenture explains. “The trust score further helps determine which user queries need to be refined for greater reliability.”
LangChain
How to monitor employee web activity?
Programmers thus discover that a language chain can be a way to automate certain extremely mundane tasks.
One example is checking web activity by employees to ensure they are not browsing illicit websites. A document describing an “acceptable use policy” for a company is uploaded to what is called a vector database, which is a special type of database that can compare strings of text from, for example, from a URL to a collection of text strings in a document to see if there is a match.
When a person types a URL into a browser, the site URL and summary of site content can be automatically compared to the policy document in the vector database to see if the site content matches prohibited topics.
The programmer can automate the comparison using a simple text prompt asking: “Is there anything on this site that matches prohibited topics?”
This example clearly shows that large language models and LangChain go beyond individual queries.
They become a way for programmers to use natural language commands to integrate the various tools at their disposal for functions that take place behind the scenes.
Towards the development of generative AI frameworks
LangChain is not the only framework for compiling workflows to make agents. Other frameworks of this type are being created, including Microsoft’s Semantic Kernel and the free software LlamaIndex, which relies on LangChain.
A group of researchers from Stanford, UC Berkeley and Carnegie Mellon presented in October what they call DSPy, a programming approach that replaces hand-written natural language prompts with functional descriptions and which can in turn automatically generate prompts. Functional descriptions can be very broad, such as “make questions and return answers”. DSPy includes a compiler that optimizes the flow of language models and supporting tools.
DSPy’s work is analogous, according to the authors, to deep learning frameworks that have advanced neural networks through layers of abstraction, including Torch and Theano.
The authors claim that the quality has improved significantly compared to manual prompt creation. “Without hand-written prompts and in just a few minutes, DSPy module compositions can increase the quality of simple programs by 33% to 82%,” they write.
Work on GenAI frameworks is still in its early stages. And you can expect many more layers of abstraction above, below and around LangChain in the coming year.
Source: “ZDNet.com”