Skip to content

Improve models docs #1274

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lukekim merged 2 commits into from
Dec 31, 2025
+120 −86
Merged

Improve models docs #1274

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b32ed3a
Improve models docs
lukekim Dec 18, 2025
eb24473
Merge branch 'trunk' into lukim/tweaks
lukekim Dec 31, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 31 additions & 35 deletions website/docs/components/embeddings/bedrock.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: 'Amazon Bedrock Model Provider'
description: 'Instructions for using Amazon Bedrock embedding models'
sidebar_label: 'AWS Bedrock'
sidebar_label: 'Amazon Bedrock'
sidebar_position: 8
---

Expand All @@ -11,33 +11,33 @@ To use an embedding model deployed to [AWS Bedrock service](https://aws.amazon.c

#### AWS Parameters

| Parameter | Description |
| ----------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `aws_region` | AWS region. Default: `us-east-1`. |
| `aws_profile` | Optional. AWS profile to use when loading credentials. |
| `aws_access_key_id` | Optional. AWS access key ID for authentication. If not provided, credentials will be loaded from environment variables or IAM roles |
| `aws_secret_access_key` | Optional. AWS secret access key for authentication. If not provided, credentials will be loaded from environment variables or IAM roles |
| `aws_session_token` | Optional. AWS session token for authentication |
| `max_concurrent_invocations` | Optional. The maximum number of concurrent API invocations. Defaults to `40` |
| `requests_per_min_limit` | Optional. The maximum number of requests made per minute. Defaults to `1500` |
| Parameter | Description |
| ---------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `aws_region` | AWS region. Default: `us-east-1`. |
| `aws_profile` | Optional. AWS profile to use when loading credentials. |
| `aws_access_key_id` | Optional. AWS access key ID for authentication. If not provided, credentials will be loaded from environment variables or IAM roles |
| `aws_secret_access_key` | Optional. AWS secret access key for authentication. If not provided, credentials will be loaded from environment variables or IAM roles |
| `aws_session_token` | Optional. AWS session token for authentication |
| `max_concurrent_invocations` | Optional. The maximum number of concurrent API invocations. Defaults to `40` |
| `requests_per_min_limit` | Optional. The maximum number of requests made per minute. Defaults to `1500` |

#### AWS Titan Models

These parameters are used for [Amazon Titan Text](https://docs.aws.amazon.com/bedrock/latest/userguide/titan-embedding-models.html) embedding model

| Parameter | Description |
| ------------ | ----------------------------------------------------------------------------------------------------------------------- |
| `normalize` | Whether or not to normalize the output embedding. Defaults to `true`. |
| `normalize` | Whether or not to normalize the output embedding. Defaults to `true`. |
| `dimensions` | The number of dimensions the output embedding should have. The following values are accepted: 1024 (default), 512, 256. |

#### Amazon Nova Models

These parameters are used for [Amazon Nova](https://docs.aws.amazon.com/nova/latest/userguide/embeddings-schema.html) multimodal embedding models

| Parameter | Description |
| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
| `dimensions` | **Required**. The number of dimensions the output embedding should have. Accepted value: 256, 384, 1024 or 3072. |
| `truncation_mode` | Optional. Specifies how the API handles inputs longer than the maximum token length. One of: `START`, `END` or `NONE` (default). |
| Parameter | Description |
| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `dimensions` | **Required**. The number of dimensions the output embedding should have. Accepted value: 256, 384, 1024 or 3072. |
| `truncation_mode` | Optional. Specifies how the API handles inputs longer than the maximum token length. One of: `START`, `END` or `NONE` (default). |
| `embedding_purpose` | Optional. Use the Nova embeddings model optimized for different purposes. Default `GENERIC_INDEX`. See reference [docs](https://docs.aws.amazon.com/nova/latest/userguide/embeddings-schema.html) for all options. |

#### Cohere Models
Expand All @@ -64,10 +64,10 @@ embeddings:
### Example `spicepod.yaml` configuration, Titan model

```yaml
- from: bedrock:amazon.titan-embed-text-v2:0
name: titan-embeddings
params:
dimensions: "256"
- from: bedrock:amazon.titan-embed-text-v2:0
name: titan-embeddings
params:
dimensions: '256'
```

### Example `spicepod.yaml` configuration, Amazon Nova model
Expand All @@ -77,7 +77,7 @@ embeddings:
- from: bedrock:amazon.nova-2-multimodal-embeddings-v1:0
name: nova-embeddings
params:
dimensions: "3072"
dimensions: '3072'
truncation_mode: START
embedding_purpose: GENERIC_RETRIEVAL
aws_region: us-east-1
Expand Down Expand Up @@ -120,7 +120,7 @@ If AWS credentials are not explicitly provided in the configuration, the connect
1. Run `aws configure sso` to configure a new SSO profile
2. Use the profile by setting `AWS_PROFILE=sso-profile`
3. Run `aws sso login --profile sso-profile` to start a new SSO session
:::
:::

3. **AWS STS Web Identity Token Credentials**:
- Used primarily with OpenID Connect (OIDC) and OAuth
Expand Down Expand Up @@ -149,25 +149,21 @@ The IAM role or user needs the following permissions to access DynamoDB tables:

```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"bedrock:InvokeModel"
],
"Resource": [
"arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-*"
]
}
]
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["bedrock:InvokeModel"],
"Resource": ["arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-*"]
}
]
}
```

### Permission Details

| Permission | Purpose |
|------------|---------|
| Permission | Purpose |
| --------------------- | --------------------------------------------- |
| `bedrock:InvokeModel` | Required. Used to invoke the embedding model. |

### Additional Information
Expand Down
2 changes: 1 addition & 1 deletion website/docs/components/embeddings/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Spice supports a variety of embedding model sources and formats:
| [`openai`][openai] | OpenAI (or compatible) LLM endpoint | Release Candidate | - | OpenAI-compatible HTTP endpoint |
| [`azure`][azure] | Azure OpenAI | Alpha | - | OpenAI-compatible HTTP endpoint |
| [`databricks`][databricks] | Models deployed to Databricks Mosaic AI | Alpha | - | OpenAI-compatible HTTP endpoint |
| [`bedrock`][bedrock] | Models deployed on AWS Bedrock | Alpha | - | OpenAI-compatible HTTP endpoint |
| [`bedrock`][bedrock] | Models deployed on Amazon Bedrock | Alpha | - | OpenAI-compatible HTTP endpoint |
| [`model2vec`][model2vec] | Model2Vec static word embeddings | Alpha | - | Model2Vec format |

[file]: /components/embeddings/local.md
Expand Down
Loading
Loading