ML experiments are hard to reproduce and track
17 posts from 15 people15d activeFading, +1 this week
Posts per day
The posts behind it
SelectorGroupChat.load_state() appends to the selector's model context instead of overwriting it (duplicate/stale history on rollback)
### Description `BaseGroupChat.load_state()` is documented to *overwrite* the current state. Participant agents get that semantics: their `ChatCompletionContext.load_state()` replaces the whole context. The selector group chat manager, however, **appends** to its...
test_source_prompt_requirements_resolves_the_name_like_load_prompt fails with sqlite 'unable to open database file'
...failed with sqlite3.OperationalError (unable to open database file) on /tmp/tmpq9i5pncm/mlflow.db. The test uses no store fixture, but since #27230 _source_prompt_requirements calls _stored_registered_model_name, which hits the global model registry store. That store points at a...
[Bug]: Failed to Load Exercise data
### Are you willing to submit a PR? _No response_ ### What happened? After updating to version 1.10, I receive an error in the Sparky web app "Error, failed to load exercise entries". However, the Android phone app (version 1.10) has the information and does not display the...
[BUG] A failed task can be skipped after checkpoint restore because its output remains set
### Description When a task produces an output but its `output_file` write fails, `TaskFailedEvent` is emitted while `Task.output` remains populated. If checkpointing is triggered by `task_failed`, the checkpoint stores that output even though the task did not emit...
My favorite ways to learn Snowflake hands-on (mostly free): a starter guide
...Sales, Marketing, Security, Product) show how they use Snowflake internally. It's less "how do I write a COPY INTO" and more "what does this look like in a real company". Examples:. Saving ~$5M in SaaS spend with AI on Snowflake. Lead scoring with Snowpark, Cortex and the Model...
[Bug]: MCP tool discovery ignores pages after nextCursor
### Bug Description `BasicMCPClient.list_tools()` and `McpToolSpec.fetch_tools()` consume only the first `tools/list` response. If a server returns `nextCursor`, later tools never reach the agent. An `allowed_tools` filter selecting a tool on the second page consequently returns...
Possible bug in Mermaid format output from the `graph` command
...is unable to identify where to place the `:::cycle` styling. ### References See this PR where I've added some tests that characterise the current Mermaid output: https://github.com/hashicorp/terraform/pull/39439#discussion_r4360441282 ### Generative AI / LLM assisted...
[BUG] Guardrail retry drops the context of the previously-successful upstream task that the retried task depends on
### Description I have a crew with two tasks, each run by its own agent: - `research_task` (researcher agent) successfully outputs the account facts, including the service code `ACCT-8841`. - - `case_note_task` (writer agent) depends on that output (context=[research_task]), has...
Flink Iceberg read fails with ClassCastException on ARRAY<STRING> IS NOT NULL predicate
### Apache Iceberg version 1.8.1 ### Query engine Flink ### Please describe the bug 🐞 ### Environment - Apache Flink 1.19.1 - `iceberg-flink-runtime-1.19-1.8.1` - Iceberg format-version 2 table - Flink batch execution mode - Iceberg REST catalog with S3-compatible storage...
Python: Bug: AnnotationContent and StreamingAnnotationContent to_element writes CitationType.X, so annotations in a rendered chat history fail to parse
### Describe the bug `AnnotationContent.to_element()` and `StreamingAnnotationContent.to_element()` store the `CitationType` enum object in the XML attribute: `CitationType` is a `str, Enum`. When the element is serialized, the attribute is written with the enum's `str()` form,...
.Net: Python: Bug: AggregatorTerminationStrategy is not a TerminationStrategy and cannot be used with AgentGroupChat
...aggregator, and the class is exported from `semantic_kernel.agents.strategies` but has no way to be plugged in. ### To Reproduce Run against current `main` (5b4a3269a): Output: The existing unit tests only call `should_terminate_async` directly, so they never exercise it through...
Python: Bug: KernelFunctionSelectionStrategy and KernelFunctionTerminationStrategy default result_parser raises TypeError
### Describe the bug `KernelFunctionSelectionStrategy` and `KernelFunctionTerminationStrategy` (python/semantic_kernel/agents/strategies) both declare a default for `result_parser` that cannot be called the way the strategy calls it. The default is a zero-argument lambda, but...
[Bug]: QdrantVectorStore numeric `EQ`/`NE` filters depend on int vs float and miss float payloads
...operator. For `NE`, the result should use `must_not` clause with a range. Obvious workaround to `EQ` is to convert all numbers as floats, but this would change value representation, which is not always acceptable. But, `NE` is not going to work anyway, so the fix is needed. ###...
...But that there's entire categories of problems that seem easy to us which LLMs really struggle with. We've stumbled on several just a few replies into casual conversation. (Can they count? Can they know if they can count? Can they reproduce results? How quickly do new...
[BUG] AsyncFileStream is rejected by file wrappers and cannot be resolved asynchronously
## Description `AsyncFileStream` is included in `FileSource` and is documented for async handles such as `aiofiles` objects. However, passing one to `TextFile` or `ImageFile` raises a validation error before it can reach `FileResolver.aresolve()`. This prevents a caller with an...
Where should train-only preprocessing state live in an MLOps stack?
...I maintain DATADOC, an open-source project focused on one part of the MLOps handoff that keeps breaking in practice: how to move preprocessing from an experiment into a repeatable inference contract.. The workflow I am testing is:. Profile the raw tabular data.. Generate a...
[FR] Expose Helm health probes as configuration variables
...and can cause false probe failures during startup or load. > #### Why is it currently difficult to achieve this use case? > The chart template currently hard-codes liveness and readiness probe timings and does not render a startup probe. Supplying similarly named keys through...
Companies and products named
About this problem
Evidence
17 posts from 15 people in 11 places, about 2 a week over 68 days. Mostly on crewAIInc/crewAI, microsoft/semantic-kernel, mlflow/mlflow. Tools named alongside: GitHub, MLflow, LlamaIndex, Apache Iceberg.
Frustration Frustration 0 of 3· Seen on GitHub issues, Reddit, Hacker News
How it was grouped
Posts that state a pain and match the "ml-reproducibility" rule. First post Sep 2, 2026, latest Nov 8, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method
The same problem elsewhere
Experiments on ClickHouse are hard to reproduce and track
8 posts from 1 person on GitHub issues. Unverified.
Experiments on DuckDB are hard to reproduce and track
6 posts from 4 people on GitHub issues.
Experiments on Codex are hard to reproduce and track
4 posts from 3 people on GitHub issues.
Experiments on LlamaIndex are hard to reproduce and track
3 posts from 3 people on GitHub issues.
Experiments on MLflow are hard to reproduce and track
3 posts from 3 people on GitHub issues.
Experiments on GitHub are hard to reproduce and track
3 posts from 3 people on GitHub issues, Reddit.
Experiments on llama.cpp are hard to reproduce and track
2 posts from 2 people on GitHub issues. Unverified.
Experiments on OpenAI are hard to reproduce and track
2 posts from 2 people on GitHub issues. Unverified.
Experiments on Apache Superset are hard to reproduce and track
2 posts from 1 person on GitHub issues. Unverified.
History