Problemsby GlyphContext windows run out on real workloads
20 posts from 20 people17d activeSteady, +2 this week
Posts per day
The posts behind it
What do you think the roles of the future look like?
...will need an "AI guy" who helps them build up a set of AI connected systems in house. . Failing us having to weld steel plates to our cars and spray paint our teeth silver, what are others thinking the future looks like?.
I try local models regularly on 2x 4090 and struggle with context length constantly. 48GB on GPU and 128GB from RAM. the original comment mentioned only 16GB of VRAM. You know talk about Open Router and 27B dense model? We are not comparing an even more expensive Hardwaresetup...
...of sycophancy, context degradation over time, or something else coding agents still struggle to verify their own work. Maybe with 1B context windows this will change. 2/ Infra is still annoying to setup. We wanted to provide a unified interface across web, mobile and soon...
Subagent skill base is the orchestrator's narrowed skill set — sub-agents can't load skills the orchestrator lacks, so orchestrators accumulate all sub-agent skills
## Component CLI - subagent / harness runtime skill resolution ## Summary When a subagent profile declares `enabled_skills` / `disabled_skills`, the subagent's available skills are resolved from the **parent (orchestrator) session's already-narrowed skill set**, filtered by the...
[Feature]: Multi-currency settlement support (non-USD cost tracking)
### Check for existing issues - [x] I have searched the existing issues and checked that my issue is not a duplicate. ### The Feature ### Problem LiteLLM currently reports all costs in USD. This makes it hard for non-US teams to track spend, set budgets, and reconcile invoices...
[Feature]: Add standard hyphenated model aliases for Cerebras Llama 3.1 models (`cerebras/llama-3.1-8b`, `cerebras/llama-3.1-70b`)
### Check for existing issues - [x] I have searched the existing issues and checked that my issue is not a duplicate. ### The Feature Please add standard hyphenated pricing and context window entries for Cerebras Llama 3.1 models in `model_prices_and_context_window.json`: -...
Cortex Agents / Snowflake CoWork: long conversations get very expensive. How are you limiting conversation memory?
...limits output generation per turn, not history input. budget.seconds : timeout. There is no way to control how much history gets sent on follow-up turns. Something like this would solve it mb:. conversation_memory: max_turns: 5 # only send the last N turns max_tokens: 500000 #...
Training a 503M language model from scratch on 3× RTX 3080s-looking for an A-Z training plan and a smaller model for testing
...hardware? Would a different depth/width balance or context length make more sense?. . How should I choose and test the learning rate? What short experiments would you run to select peak LR, warmup, decay, and effective batch size? What loss or gradient behavior tells you the...
Le Bûcher de Bohemian Grove: Meta’s Muse-Sparck-816B MoE Abattoir
...pas du machine learning, c'est du bricolage de désespoir. . Then comes the projector nightmare. Your language backbone sweats blood to build a deep 8192-dimensional semantic manifold, only for you to violently amputate it at the exit. out_hidden_size: 6144 against a hidden_size:...
include_team_models=true returns no metadata for a team service-account key with working inference access
## Environment - LiteLLM database image `v1.105.0` - Team service-account key - Team explicitly granted the relevant model aliases ## Description The same team service-account key can invoke its permitted models and retrieve their unfiltered metadata, but...
...can read more about that here https://www.linum.ai/about if you're curious). The biggest bottleneck for our last text-to-video model in terms of training and inference cost is attention. Video models are incredibly token dense (e.g. 110K tokens for a several second clip). If we...
...with Opus 5.5. But it also works fine with GPT 6.1 Sol, Kimi K3 and MiMo 2.6 Pro. It doesn't work equally well with Sonnet 5.5, interestingly.
Feature request: allow lowering max context window to fit bigger models
I have a 32GB M1 Max. On LMStudio I can run Qwen3.8 27B 4 bit quant by limiting the context window to below the max of 262144. The model itself is around 16GB Magnitude won't allow me to install it because it says I need 5GB more ram. I suspect because it's accounting for the...
How to find out the "Output token limit" of LLMs in GitHub Copilot?
...limits. It is unlikely that this raw limit will be a available inside GitHub Copilot.. Is there any way to find out the actual/ usable output token limit of models inside GitHub Copilot?.
[Bug]: Databricks model map needs to be updated for Unity Gateway
### Description Your [model map](https://raw.githubusercontent.com/BerriAI/litellm/refs/heads/main/model_prices_and_context_window.json) currently has model names of the form `databricks/databricks-claude-sonnet-5`. Databricks has updated their internal routing, and using these...
Why doen't AI companies agree on a single standard?
...them. I'm new to local models after using cloud models for years which "just works". I hate how many different engines, api formats, harnesses, tools I had to go through to optimise my setup. Maybe I'm missing something..
...they work). And of course that can happen from random chance, and of course the LLM has no way to externally verify the extent to which randomness is in play (or the ground-truth probabilities). The paper makes clear that they used pre-trained, frontier models - in other words,...
What is the general design of these new math solving systems? [D]
...and see if it can answer a question I have about higher dimensional geometry. I'm struggling to find a meaningful way to compose larger ideas from smaller ones. I can imagine it's relatively simple if you know what to do. . What things have you seen? Do you have any ideas you...
> Where are all these verbal tics coming from and why is it so hard to get rid of them? It’s a side effect of post-training for effectiveness and efficiency at technical tasks. Over time the models learn to pack as much information as possible into their available context...
Companies and products named
About this problem
Evidence
20 posts from 20 people in 10 places, about 2 a week over 68 days. Mostly on Hacker News comments, BerriAI/litellm, r/learnmachinelearning. Tools named alongside: BerriAI, Databricks, Claude, LM Studio.
Frustration Frustration 1 of 3· Seen on Hacker News, Reddit, GitHub issues
How it was grouped
Posts that state a pain and match the "context-window" rule. First post Sep 1, 2026, latest Nov 7, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method
The same problem elsewhere
Claude Code runs out of context on real work
12 posts from 12 people on GitHub issues, Reddit, Hacker News.
Claude runs out of context on real work
10 posts from 8 people on GitHub issues, Reddit, Hacker News.
OpenAI runs out of context on real work
8 posts from 8 people on GitHub issues, Community forums.
ChatGPT runs out of context on real work
8 posts from 8 people on GitHub issues, Reddit, Hacker News.
Ollama runs out of context on real work
7 posts from 7 people on GitHub issues, Reddit.
Codex runs out of context on real work
6 posts from 6 people on GitHub issues, Reddit, Hacker News.
Cursor runs out of context on real work
5 posts from 5 people on Community forums, Reddit, Hacker News.
vLLM runs out of context on real work
5 posts from 5 people on GitHub issues.
Mistral runs out of context on real work
4 posts from 4 people on GitHub issues.
Anthropic runs out of context on real work
4 posts from 4 people on GitHub issues.
llama.cpp runs out of context on real work
4 posts from 4 people on GitHub issues.
Hugging Face runs out of context on real work
3 posts from 3 people on GitHub issues.
History