Prompt Perfection Proven Pointless

Why your casual ChatGPT chats are fine (and other revelations that'll make you feel better)

Hello AI Explorers! ๐Ÿš€ This week's newsletter unpacks everything from Google's latest learning companion to the surprising revelation that AI might need therapy someday. Time to decode this week's AI adventures, only your own biological neural network is required.
Highlights from this weekโ€™s edition

  • ๐ŸŽฏ Prompt Perfect?: New research reveals when fancy prompting matters (spoiler: your casual ChatGPT chats are fine!)

  • ๐ŸŽจ Gamma Spotlight: Meet your new presentation buddy that tries to make the design-challenged look like pros

  • ๐Ÿงฎ Scale Check: Has AI hit its growth ceiling? The latest buzz about LLM scaling limitations and what it means for education

  • ๐ŸŽ“ Chegg Check: The homework helper platform lost 500k subscribers to ChatGPT (pour one out for 99% of their stock value)

  • ๐Ÿ” Google's New Toy: Learn About launches in the US (VPN friends unite!). Will self-directed learning be as easy as clicking "next"

  • ๐ŸŽค Epic Listen: Lex Fridman's 5-hour AI deep dive with Anthropic's dream team (because who doesn't want to hear about AI-obsessed bridges?)

  • ๐ŸŒฒ Story Time: The tale of Elara and her forest adventures - ChatGPT's favorite bedtime story

Prompt of the Week ๐Ÿ’ญ

Good prompting doesn't always matter. Sometimes you just want a simple answer to a question and you don't need to give the LLM (large language model) a bunch of context and background information. Sometimes you just want to have a back and forth conversation which itself will build up context. This paper shows that good prompting really doesn't matter for everything and identified some places where good prompting is important. Testing was done on Claude-3.5-sonnet, GPT-4o (chatGPT), Llama3 (from Meta) and Qwen1.5 (from Alibaba)

It turns out that good prompting doesn't really matter for:

  • strategic and technical business solutions

  • expert technical guidance in specialized domains

  • IT and computing problem solving
     

But it does matter for:

  • technical coding and data visualization

  • programming and scripting tasks

  • algorithmic problem solving

  • applied technical and analytical problem solving

  • creative tasks

AI App Spotlight ๐Ÿ”

I haven't had a ton of time to play with it, but I'm really liking Gamma. It's a presentation app that takes your content or even just a prompt and creates presentations, documentation, or websites. I haven't really been a fan of specialized AI tools like this since a lot of the time, everything the specialized tools can do, you can do in ChatGPT or Claude. Gamma has changed my mind.

I have very little design skill, so a tool that can take over for me on that end is super valuable. I use Canva a lot for that, but Gamma goes a step further for me. With Canva, I still have to know colour schemes and stuff like that. There are some good templates for that, but it still requires me to pick what I think looks best, which is definitely not going to be what most people think looks best. Gamma helps with this because I can just give it a theme that I am going for and it will create a slide deck that matches that theme.

Gamma follows the freemium model and the paid plan offers decent basic features, a limited number of design credits and Gamma-branded exports. $10USD/month gives a bunch more credit and removes the Gamma branding.

AI News of the Week ๐Ÿ“ฐ

๐Ÿ“ฐ

  • Has generative AI hit a scaling wall? In other words, have improvements in LLMs slowed down/halted because training on more and more data doesn't lead to better LLMs? This could well be true, but even if it is, I don't see this stopping progress with AI because there are still other routes to improve LLM performance (e.g., let them "think" for longer like with OpenAIs o1-preview) or provide new interfaces (e.g., Claude's computer use)
     

  • A new math benchmark for AI that might actually hold up over time
     

  • AI companies (well, Anthropic is anyway) are starting to take seriously the possibility that AI systems will become conscious. This research firm believes that if AI can become conscious, then the moral thing to do. is to plan for that eventuality.
     

  • You may or may not know podcaster Lex Fridman. You may or may not like podcaster Lex Fridman, but somehow he is able to get really interesting people on his podcast. In this 5 hour ๐Ÿ˜ฑ podcast (I've listened to about 1/3 of it so far) Amodei talks about the importance of AI safety and how Anthropic puts that at the center.

Askell is a philospoher by training and is now involved in developing Claude's personality and capability. She talks about the challenges of creating a system that is accurate and honest, asks follow-up questions without being annoying, and is not overly agreeable but sill culturally adaptable

I haven't gotten to the Chris Olah part of the interview yet, but he is an expert on mechanistic interpretability which maybe sounds kinda awful, but I find it fascinating. It focuses on trying to understand how generative AI actually creates its responses. It's kinda like being a neuroscientist, but for generative AI. One interesting project that Anthropic did this year was "Golden Gate Claude" where they basically made a version of Claude obsessed with the Golden Gate Bridge so that every response it generated was associated with the bridge somehow.

Watch on Youtube or listen wherever you get your podcasts (Lex Fridman episode 452)

AI Powered Pedagogy ๐Ÿ‘ฉโ€๐Ÿซ

  • Upcoming Webinars (You may find these seminars interesting, but I don't have any insight into how good or informative they will be for you)

The mAIn Event

Google's "Learn About" ๐ŸŽ“

Following the success of NotebookLM, Google has launched Learn About, an AI powered educational resource that is supposed to enhance the learning experience by providing a structured and interactive format for learning about a topic.

Learn About is in the experimental phase and you can try it out by following the link below, but don't click on the link yet because you need to be in the US to use it. Or at least Google needs to think you are in the US, so you can just use a VPN to access it. The easy way that I do this is just use the Opera browser which has a built in VPN that you can point to the US.

How it works

  • Enter any topic (like "How is AI affecting higher education?")
     

  • Click through interactive lists to explore subtopics
     

  • Follow your curiosity without complex AI prompting
     

  • Get visual, organized content from reliable sources 

The big picture

While other AI tools require expertise and careful prompting, Learn About aims to improve upon this by:

  • Handling complex AI interactions behind the scenes
     

  • Creating clear, structured learning paths
     

  • Reducing information overload
     

  • Making self-directed learning accessible to anyone with internet

Yes, but some limitations exist

  • Sessions can't be saved (yet)
     

  • Standard AI issues like bias remain
     

  • Google's history of discontinuing projects raises sustainability questions

What's next

While officially waitlisted, most Google account holders can access Learn About now (if you use a VPN to the US). Educators should test it as a potential tool to boost student agency and independent learning skills.

The bottom line

Learn About is aiming for the sweet spot between encyclopedia simplicity and text book depth, making it a promising tool for giving studentโ€™s some autonomy in their learnign in higher education.

Reply

or to participate.