# Siren AI Plugin

This Siren AI plugin includes following features:

- Generate AI graph reports
- Siren API methods for interacting with LLMs

## Usage

See [our official docs](docs/modules/siren-ai/pages/c_introduction.adoc) for setting up and using the plugin.

## Developer setup
1. Install dependencies:
   ```bash
   yarn
   ```
2. Configure Investigate YAML with one of our internal LLMs (see [LLM setup for developers](#llm-setup-for-developers))
3. Install the plugin into your locally running Investigate environment:
   ```bash
   yarn dev path/to/kibi-internal
   ```
   This command then waits for any code changes and syncs them as they happen.
4. Start Investigate

## LLM setup for developers

Internally, we have a couple of Azure OpenAI deployments and some AWS Bedrock models available to developers.

<details>
<summary>Azure OpenAI</summary>

Siren's Azure OpenAI deployment information and credentials found in 1Password:
- `product-ai-test-2`
- `product-ai-test-3`
</details>

<details>
<summary>AWS Bedrock</summary>

To check your access to LLMs on AWS Bedrock, follow these steps
1. Go [here](https://d-90676e498f.awsapps.com/start#/), expand the "Devops" dropdown and check if you have the `bedrock-full-access` account
2. If you do see `bedrock-full-access`, click it to sign in with that account
3. Click the region at the top right and make sure you're on "US East (Ohio)" (`us-east-2`)
4. Search for the Amazon Bedrock app and then open the Chat/Text Playground
5. Click "Select model" and under the "Meta" category, you should see on LLM that is available ("Llama 3.3 70B Instruct")
6. Try to chat with it! If you get a permissions error, ask Andrew to give you the `bedrock:InvokeModel` and `bedrock:InvokeModelWithResponseStream` permissions

Now that you have access to the LLMs, you can provide credentials to our AI client in the following ways:
- Specifying them explicitly using the `providerConfig.aws.connection.credentials` entry (you can find these in the [AWS access portal](https://d-90676e498f.awsapps.com/start))
- Using environment variables: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN` (optional), `AWS_CREDENTIAL_EXPIRATION` (optional)
- From your AWS CLI config file (works automatically if logged in through the AWS CLI)

See [this example AWS config](docs/modules/siren-ai/pages/t_configuring_aws.adoc) for what your config may look like. If you want to use credentials from the AWS CLI instead of explicitly providing them in the Investigate YAML, do the following:
1. [Install AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) if not already installed (must be version 2.x.x)
2. Initialize authentication with `aws configure sso`. The details should be as follows:
   - SSO session name: `main` (You can choose any name you want here)
   - SSO start URL: `https://d-90676e498f.awsapps.com/start`
   - SSO region: `us-east-1`
   - SSO registration scopes: (don't input anything; leave default)
   - Role name: `bedrock-full-access`
   - CLI default region `us-east-2`
   - CLI default output format: `None`
   - CLI profile name: `bedrock-user`
3. Then try invoking the LLM using the AWS CLI with this newly created profile:
   ```shell
   aws --profile bedrock-user bedrock-runtime converse --model-id 'meta.llama3-3-70b-instruct-v1:0' --messages '[{"role": "user", "content": [{"text": "Hello!"}]}]'
   ```
   If this works, your credentials are set up to use AWS Bedrock! Update your Investigate YAML to use the correct profile and it should work automatically. The region key will be optional because we set the default region for this profile.
</details>

## Developer info

### Data structures

Data is represented and passed between different services and components as an array of items, specifically, chat messages, tool calls and tool responses. When you send a message from the frontend, that data item goes through a few transformations before it reaches the actual LLM. For example, when using OpenAI and its Responses API, the messages go through these structures:

| Location                | Type                                               | Notes                                       |
|-------------------------|----------------------------------------------------|---------------------------------------------|
| Frontend                | [`Message[]`](src/common/types.ts)                 | Type that has information useful to the UI  |
| Backend                 | [`Message[]`](src/common/types.ts)                 | Uses `convertMessageToLlmInput()`           |
| AI client (interface)   | [`LlmInput[]`](src/server/ai_client/ai_client.ts)  | Generic type without UI-specific properties |
| OpenAI client           | `ResponseInputItem[]`                              | Uses `convertLlmInputToOpenAiInput()`       |
| _(LLM processes input)_ | _Some HTTP format_                                 |                                             |
| OpenAI client           | `ResponseOutputItem[]`                             | Uses `convertOpenAiOutputToLlmOutput()`     |
| AI client (interface)   | [`LlmOutput[]`](src/server/ai_client/ai_client.ts) |                                             |
| Backend                 | [`Message[]`](src/common/types.ts)                 | Uses `convertLlmOutputToMessage()`          |
| Frontend                | [`Message[]`](src/common/types.ts)                 |                                             |
