Repository navigation
refactor(llm): migrate model parameters to typed struct - #80
Merged
Merged
Conversation
Introduce a new `Parameters` struct to replace the untyped `HashMap<String, Value>` for model parameters. This provides type safety and better validation for common parameters like `max_tokens`, `temperature`, `top_p`, `top_k`, and `reasoning`. The `Parameters` struct includes strongly-typed fields for standard parameters and an other field for provider-specific options. It also provides a `merge()` method for combining parameter sets and a `set()` method for parsing string values from the command-line. Update all providers to use the typed fields directly instead of parsing from JSON values. This eliminates runtime type casting and improves error handling through the new `SetParameterError` type. Undefined model parameters can still be set through the `other` field, which is a `HashMap<String, Value>`. For example, `Ollama` has a `keep_alive` parameter that is specific to that provider, and is not exposed as a typed field. Signed-off-by: Jean Mertz <git@jeanmertz.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduce a new
Parametersstruct to replace the untypedHashMap<String, Value>for model parameters. This provides type safety and better validation for common parameters likemax_tokens,temperature,top_p,top_k, andreasoning.The
Parametersstruct includes strongly-typed fields for standard parameters and an other field for provider-specific options. It also provides amerge()method for combining parameter sets and aset()method for parsing string values from the command-line.Update all providers to use the typed fields directly instead of parsing from JSON values. This eliminates runtime type casting and improves error handling through the new
SetParameterErrortype.Undefined model parameters can still be set through the
otherfield, which is aHashMap<String, Value>. For example,Ollamahas akeep_aliveparameter that is specific to that provider, and is not exposed as a typed field.