re:Invent 2023 day 3 recap

AI-related announcements dominate once more. New models for Bedrock. Vector search everywhere, and DynamoDB does a bait-and-switch. Welcome to day 3 of re:Invent 2023.

Bedrock FMs goes brrr…

Several foundational models are now available on Bedrock, including:

• Anthropic Claude 2.1
• Amazon Titan Text models (Lite & Express)
• Amazon Titan Multimodal Embeddings model
• Amazon Titan Image Generator model
• Stable Diffusion XL 1.0
• Meta Llama 2 70B

With the recently discovered vulnerability in ChatGPT and rumours of its coding ability being nerfed. All these new models have arrived at a good time.

Bedrock supports batch inference

Here’s the official announcement.

You can now process prompts in Bedrock in batches. I’m not sure of the use case for this…

Is this for when autonomous agents perform user-requested actions in the background?

Or perhaps for high-throughput AI applications, so you group user prompts in mini batches to reduce the no. of API requests?

SageMaker gets HyperPod

Here’s the official announcement.

For anyone who needs to train their own LLM models, SageMaker HyperPod promises to reduce training time by a 40%.

That’s huge! Training LLM models can take weeks at a time. If HyperPod delivers on the promise, it will almost double the speed of iterations.

Vector search everywhere

Vector databases are becoming a big part of AI development. As many of us predicted, AWS announced vector support for a number of its databases, including:

• Vector engine for OpenSearch Serverless is GA (remember, OpenSearch Serverless has a minimum charge of $700/month)
• Vector search for Amazon MemoryDB for Redis
• Vector search for Amazon DocumentDB

When Dr. Swami Sivasubramanian said “and DynamoDB”, my heart stopped for a moment. Really? DynamoDB gets vector search? How did they even make it work?

But no, you get vector search for DynamoDB by sending your DynamoDB data to OpenSearch first…

You can do it with the no-code integration we covered in the day 2 recap.

OpenSearch gets new instance family

Here’s the official announcement.

OpenSearch is getting a lot of attention at this re:Invent for its proximity to AI development. There’s a new OpenSearch Optimized Instance family, called OR1. It promises a 30% better price-to-performance ratio.

Neptune gets new analytics engine

Here’s the official announcement.

Graph databases have a lot of use cases. We’ve seen them used to detect fraud, model game economies and analyze the Panama paper. With the new analytics engine, you can run graph analytics on data in your existing Neptune graph databases or graph data in S3.

I’m excited to find out what you can do with this new capability.

AWS Marketplace gets new API

Here’s the official announcement.

This is an odd one amongst all the AI-releated announcements the last two days. I know a lot of SAAS companies who sell their products on the AWS Marketplace and this new API should help with their orchestration and hopefully create a better end-user experience too.