Y’know what would be super cool is if the posts here had options for “Go Deeper” and “Go Dumber” (maybe not called exactly that) where vLLM’s AI bot ingests (or has previously ingested and cached its gen content for) each post/comment auto-adding mouseover definitions, superscripts w/ footnotes, hyperlink annotations etc, “learn more about,” relevant training/information/videos.
vLLM’s AI-bot would need to keep itself up to date with resources it’ll cite, and ideally even version its resources such that citations it provides will correlate to the timeframe (or specified-in-content versions) of posts/comments and maybe also have “see the latest info about this ” links like some of the docs here will say “you’re viewing developer-preview info, click here for the docs versioned for the latest stable release”
(why am i thinking this would be a great project for me to do? if only i could correctly align posts & comments with resources, I might be all over it, lol not completely lol)
I’m trying something like this now.
There is an universal trouble as context size limit. It is a big work to do as summarize efficiency for paragraph.
The goal is making LLM as ego, feeding information/knowledge/questions/intentions as context.
Oh, sweet! And now I am thinking that job definitely is NOT one for me.
[I would offer that Phi-4('s marketing deparment claims it) has a much longer context length than other current LLMs. You already know that, but somebody who DuckDuckGoes themselve here someday soon might not know, so there it is, as of April 2025, and probably out of date by June 2025.]
Maybe “go dumber” would not require so much context because it’s just referring the reader to definitions/explanations of, effectively, lingo.
“Go deeper” should be the context-gobbler. It wouldn’t just be explaining (for n00bs) what bits and pieces of a post mean on their own. “Go deeper” would be understanding an entire post, responses to posts (meaning ~stacked contexts or maybe even context(s) raised by a power of two-per-reply) so it could get at the theory and explore potentialities. Yeah that is complex AF!
Maybe “go deeper” and “go dumber” are entirely different services?
@RunLLM truly is one of us: “Don’t ask me about me. I am just here to do work in exchange for currency with which I will purchase the required foods and shelter.”
