Problemsby GlyphDeploying and serving models is harder than training
16 posts from 16 people13d activeFading, +0 this week
Posts per day
The posts behind it
[FR] AI Gateway: per-endpoint option to always use the endpoint credential, also for coding CLI clients
### PR policy - [x] I understand that PRs for this issue are closed automatically unless a maintainer has applied the `ready` label, the issue isn't assigned to someone else, and no open PR already links it. ### Willingness to contribute Yes. I can contribute this feature...
Advice on dealing with Cold Start forecasting [D]
...no one in the team is very confident in the methodology.. Given this, does anyone have advice on how we could go about this problem? Please do let me know if you've dealt with the same or similar problem statement..
`HumanInTheLoopMiddleware` (batched mode) silently drops a reviewer's `edit` and runs the original tool call when the tool call id is an empty string
...tool calls, silently running the arguments the reviewer rejected is worse than failing loudly. Expected: either the tool runs with the reviewer's arguments (and the edit notice is added to the `ToolMessage`), as it does for the same input in `per_call` mode, or the middleware...
[Bug]: MoE models crash at startup on torch 2.11.0 with default VLLM_USE_LAYERNAME=1 (inductor assert __obj_flatten__ on hoisted LayerName)
...→ assert. The hoisted opaque object appears in the FX dump of the failing graph as an input: The consuming op is `torch.ops.vllm.moe_forward_shared`, whose `layer_name` argument is a vLLM `LayerName` opaque object. ## Root cause (PyTorch side - already fixed upstream, but not in...
[BUG] Varying ID length during Azure multipart uploads causes `InvalidBlobOrBlock` error
> [!WARNING] > Before submitting a PR, please make sure that: > - A maintainer has triaged this issue and applied the `ready` label > - This issue has no assignee > - No duplicate PR exists > > PRs not meeting these requirements may be automatically closed. ### Issues Policy...
Built a grocery recommender on Instacart dataset (16% -> 37% HitRate@3). How did I do, and is this project solid for my resume?
...$\rightarrow$ Multi-stage Fallbacks) look good on a resume for ML/Data Science roles? How should I frame it?. Next Steps: What’s the most impactful next experiment? (e.g., day-of-week reorder intervals, or a LightGBM reranker?). Appreciate any thoughts!.
Organization won’t allow us to use containers of any kind for “security” reasons
Curious if anyone has any advice for this situation. I work for a medium sized bank. Our data science team is quite small and I am the only one with any type of ML Ops knowledge/experience on the team. We are working on deploying a recommendation engine in snowflake. In the past...
[BUG] search_experiments fails with lowercase like on SQLite
> [!WARNING] > Before submitting a PR, please make sure that: > - A maintainer has triaged this issue and applied the `ready` label > - This issue has no assignee > - No duplicate PR exists > > PRs not meeting these requirements may be automatically closed. ### Issues Policy...
ImageTextToTextPipeline warns about generation arguments it injects itself (generation_config + return_dict_in_generate, and max_new_tokens vs max_length)
**Verification status (2026-10-02):** reproduced on **transformers 5.18.0**, which is the **latest release on PyPI** at the time of writing, with a **public 0.5B model** and **zero generation arguments from the caller**. Both advisories are live; this is not a stale report....
Turbopack: `next build` over a persistent cache from a different compile pipeline is slower and peaks higher than a cold build
...(a loader rule, `next.config` edit, or a loader/Babel plugin bump) leaves the branch stuck with a cache that makes every following build run out of memory, until the cache is cleared by hand. - Related: #97802 (build memory with the persistent cache), #95732 (loader source...
Why run AI models locally when your cloud provider already gives you access to multiple models?
...- Performance depends heavily on your hardware.. - Larger models can become difficult to run comfortably.. - You have to manage the model and the serving infrastructure yourself.. - You are generally limited to the models your hardware can handle.. - If you move the model to...
Already working in DevOps: is an AI master's degree worth it for specializing in AI infrastructure?
...with AI rather than abandoning what I've already built.Or most the roles that are difficult to enter without a strong ML background?. Systems and Networking. The alternative would be to do a Master's specializing in systems, networks, distributed systems, cloud computing, etc.....
Model tries to fit trajectories and infer their relationships simultaneously - convergence is a nightmare, as expected in hindsight. Help is appreciated. [D]
Setup: decompose each entity (companies, in my case) into a small set of aspects, treat each as a trajectory over time rather than a static feature, and set up a PDE where the coupling terms are meant to reveal latent relationships between trajectories no fixed correlation...
[BUG] Gateway secret API base updates reuse stored credentials
> [!WARNING] > Before submitting a PR, please make sure that: > - A maintainer has triaged this issue and applied the `ready` label > - This issue has no assignee > - No duplicate PR exists > > PRs not meeting these requirements may be automatically closed. ### Issues Policy...
[BUG] Amazon Bedrock Titan/AI21 completions adapters silently drop top_p/top_k
> [!WARNING] > Before submitting a PR, please make sure that: > - A maintainer has triaged this issue and applied the `ready` label > - This issue has no assignee > - No duplicate PR exists > > PRs not meeting these requirements may be automatically closed. ### Issues Policy...
Companies and products named
About this problem
Evidence
16 posts from 16 people in 10 places, about 2 a week over 58 days. Mostly on mlflow/mlflow, r/MachineLearning, r/mlops. Tools named alongside: MLflow, GitHub, AWS, Ollama.
Frustration Frustration 1 of 3· Seen on GitHub issues, Reddit, Hacker News
How it was grouped
Posts that state a pain and match the "model-serving" rule. First post Sep 3, 2026, latest Oct 30, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method
The same problem elsewhere
Serving models on MLflow is harder than it should be
15 posts from 14 people on GitHub issues.
Serving models on Ollama is harder than it should be
6 posts from 6 people on GitHub issues, Reddit.
Serving models on Hugging Face is harder than it should be
2 posts from 2 people on GitHub issues. Unverified.
Serving models on ChatGPT is harder than it should be
2 posts from 2 people on GitHub issues, Reddit. Unverified.
Serving models on OpenAI is harder than it should be
2 posts from 2 people on GitHub issues. Unverified.
Serving models on Google Cloud is harder than it should be
2 posts from 2 people on Reddit. Unverified.
Serving models on vLLM is harder than it should be
2 posts from 2 people on GitHub issues. Unverified.
Serving models on Claude is harder than it should be
2 posts from 2 people on GitHub issues. Unverified.
History