Subscribe
Sign in
Home
Audio
Interviews
Leaderboard
About
Latest
Top
Discussions
How RLHF works, part 2: A thin line between useful and lobotomized
Many, many signs of life for preference fine-tuning beyond spoofing chat evaluation tools.
May 1
•
Nathan Lambert
20
Share this post
How RLHF works, part 2: A thin line between useful and lobotomized
www.interconnects.ai
Copy link
Facebook
Email
Note
Other
April 2024
Phi 3 and Arctic: Outlier LMs are hints
Models that seem totally out of scope from recent open LLMs give us a sneak peek of where the industry will be in 6 to 18 months.
Apr 30
•
Nathan Lambert
11
Share this post
Phi 3 and Arctic: Outlier LMs are hints
www.interconnects.ai
Copy link
Facebook
Email
Note
Other
AGI is what you want it to be
Certain definitions of AGI are backing people into a pseudo-religious corner.
Apr 24
•
Nathan Lambert
24
Share this post
AGI is what you want it to be
www.interconnects.ai
Copy link
Facebook
Email
Note
Other
3
Llama 3: Scaling open LLMs to AGI
Llama 3 shows that scaling won't be a limit for open LLM progress in the near future.
Apr 18
•
Nathan Lambert
43
Share this post
Llama 3: Scaling open LLMs to AGI
www.interconnects.ai
Copy link
Facebook
Email
Note
Other
Stop "reinventing" everything to solve alignment
Integrating some non-computing science into reinforcement learning from human feedback (RLHF) can give us the models we want.
Apr 17
•
Nathan Lambert
14
Share this post
Stop "reinventing" everything to solve alignment
www.interconnects.ai
Copy link
Facebook
Email
Note
Other
The end of the “best open LLM”
Modeling the compute versus performance tradeoff of many open LLMs.
Apr 15
•
Nathan Lambert
34
Share this post
The end of the “best open LLM”
www.interconnects.ai
Copy link
Facebook
Email
Note
Other
We disagree on what open-source AI should mean
How to read what multiple people mean by the word openness and see through the PR speak.
Apr 3
•
Nathan Lambert
20
Share this post
We disagree on what open-source AI should mean
www.interconnects.ai
Copy link
Facebook
Email
Note
Other
March 2024
DBRX: The new best open model and Databricks’ ML strategy
Databricks’ new model is surpassing the performance of Mixtral and Llama 2 70B while still being in a size category that's reasonably accessible.
Mar 28
•
Nathan Lambert
34
Share this post
DBRX: The new best open model and Databricks’ ML strategy
www.interconnects.ai
Copy link
Facebook
Email
Note
Other
1
Evaluations: Trust, performance, and price (bonus, announcing RewardBench)
Evaluation is not only getting harder with modern LLMs, it’s getting harder because it means something different.
Mar 20
•
Nathan Lambert
19
Share this post
Evaluations: Trust, performance, and price (bonus, announcing RewardBench)
www.interconnects.ai
Copy link
Facebook
Email
Note
Other
Model commoditization and product moats
Where moats are tested now that so many people have trained GPT4 class models. Claude 3, Gemini 1.5, Inflection 2.5, and Mistral Large are here to…
Mar 13
•
Nathan Lambert
24
Share this post
Model commoditization and product moats
www.interconnects.ai
Copy link
Facebook
Email
Note
Other
The koan of an open-source LLM
A proposal for a new definition of an “open-source” LLM and why no definition will ever just work.
Mar 6
•
Nathan Lambert
19
Share this post
The koan of an open-source LLM
www.interconnects.ai
Copy link
Facebook
Email
Note
Other
Interviewing Louis Castricato of Synth Labs and Eleuther AI on RLHF, Gemini Drama, DPO, founding Carper AI, preference data, reward models…
An interview I've wanted to bring you for a while.
Mar 4
•
Nathan Lambert
16
Share this post
Interviewing Louis Castricato of Synth Labs and Eleuther AI on RLHF, Gemini Drama, DPO, founding Carper AI, preference data, reward models, and everything in between
www.interconnects.ai
Copy link
Facebook
Email
Note
Other
1:26:27
Share
Copy link
Facebook
Email
Note
Other
This site requires JavaScript to run correctly. Please
turn on JavaScript
or unblock scripts