Since OpenAI launched ChatGPT, many companies have attempted to create their own AI models. However, only a few have managed to stand out. Anthropic is one of them.
The AI startup launched its own AI model, Claude, in March. It turned out to be a worthy rival to OpenAI’s GPT-3.5 and GPT-4 models. During this first launch, Anthropic also launched Claude Instant, a lighter, cheaper and faster version of Claude, according to Anthropic. Today it gets an update.
On Wednesday, Anthropic released Claude Instant 1.2, an improved version of the model that builds on Claude 2.0, the latest version of Claude released in July.
Because it uses the advanced capabilities of Claude 2.0, Claude Instant 1.2 is significantly improved in math, coding, reasoning and security, and generates longer, more structured responses, according to the release.
To put the model to the test, Anthropic compared the performance of Claude Instant 1.1 and 1.2 in standard benchmark assessments, including the Codex Assessment and the Math Problem Assessment (GSM8k), which are good benchmarks for computing and coding skills.
In both cases, 1.2 outperformed 1.1 with a score of 58.7% vs. 52.8% for the original in Codex evaluation and 86.7% vs. 80.9% for the original in GSM8k.
For other benchmark tests, the new model scored slightly lower or better than the older model, with minimal differences.
The quality of responses also improved, with fewer hallucinations and increased resistance to jailbreaking attempts. An evaluation in red team mode showed that Claude 1.2 is the safest model to use.
Companies can access the new model by filling out an interest form and developers can use the API, which is much cheaper than Claude 2.
Source: ZDNet.com