Skip to content

refactor(llm): migrate model parameters to typed struct - #80

Merged
JeanMertz merged 1 commit into
mainfrom
jean/structured-parameters
May 23, 2025
Merged

JeanMertz merged 1 commit into
mainfrom
jean/structured-parameters

Conversation

@JeanMertz

Copy link
Copy Markdown
Collaborator

Introduce a new Parameters struct to replace the untyped HashMap<String, Value> for model parameters. This provides type safety and better validation for common parameters like max_tokens, temperature, top_p, top_k, and reasoning.

The Parameters struct includes strongly-typed fields for standard parameters and an other field for provider-specific options. It also provides a merge() method for combining parameter sets and a set() method for parsing string values from the command-line.

Update all providers to use the typed fields directly instead of parsing from JSON values. This eliminates runtime type casting and improves error handling through the new SetParameterError type.

Undefined model parameters can still be set through the other field, which is a HashMap<String, Value>. For example, Ollama has a keep_alive parameter that is specific to that provider, and is not exposed as a typed field.

Introduce a new `Parameters` struct to replace the untyped
`HashMap<String, Value>` for model parameters. This provides type safety
and better validation for common parameters like `max_tokens`,
`temperature`, `top_p`, `top_k`, and `reasoning`.

The `Parameters` struct includes strongly-typed fields for standard
parameters and an other field for provider-specific options. It also
provides a `merge()` method for combining parameter sets and a `set()`
method for parsing string values from the command-line.

Update all providers to use the typed fields directly instead of parsing
from JSON values. This eliminates runtime type casting and improves
error handling through the new `SetParameterError` type.

Undefined  model parameters can still be set through the `other` field,
which is a `HashMap<String, Value>`. For example, `Ollama` has a
`keep_alive` parameter that is specific to that provider, and is not
exposed as a typed field.

Signed-off-by: Jean Mertz <git@jeanmertz.com>
@JeanMertz
JeanMertz merged commit 1e52932 into main May 23, 2025
@JeanMertz
JeanMertz deleted the jean/structured-parameters branch May 23, 2025 09:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant