- There Is No Spoon
- Posts
- Prompt Perfection Proven Pointless
Prompt Perfection Proven Pointless
Why your casual ChatGPT chats are fine (and other revelations that'll make you feel better)
Hello AI Explorers! ๐ This week's newsletter unpacks everything from Google's latest learning companion to the surprising revelation that AI might need therapy someday. Time to decode this week's AI adventures, only your own biological neural network is required.
Highlights from this weekโs edition
๐ฏ Prompt Perfect?: New research reveals when fancy prompting matters (spoiler: your casual ChatGPT chats are fine!)
๐จ Gamma Spotlight: Meet your new presentation buddy that tries to make the design-challenged look like pros
๐งฎ Scale Check: Has AI hit its growth ceiling? The latest buzz about LLM scaling limitations and what it means for education
๐ Chegg Check: The homework helper platform lost 500k subscribers to ChatGPT (pour one out for 99% of their stock value)
๐ Google's New Toy: Learn About launches in the US (VPN friends unite!). Will self-directed learning be as easy as clicking "next"
๐ค Epic Listen: Lex Fridman's 5-hour AI deep dive with Anthropic's dream team (because who doesn't want to hear about AI-obsessed bridges?)
๐ฒ Story Time: The tale of Elara and her forest adventures - ChatGPT's favorite bedtime story
Prompt of the Week ๐ญ
Good prompting doesn't always matter. Sometimes you just want a simple answer to a question and you don't need to give the LLM (large language model) a bunch of context and background information. Sometimes you just want to have a back and forth conversation which itself will build up context. This paper shows that good prompting really doesn't matter for everything and identified some places where good prompting is important. Testing was done on Claude-3.5-sonnet, GPT-4o (chatGPT), Llama3 (from Meta) and Qwen1.5 (from Alibaba)
It turns out that good prompting doesn't really matter for:
strategic and technical business solutions
expert technical guidance in specialized domains
IT and computing problem solving
But it does matter for:
technical coding and data visualization
programming and scripting tasks
algorithmic problem solving
applied technical and analytical problem solving
creative tasks
AI App Spotlight ๐
I haven't had a ton of time to play with it, but I'm really liking Gamma. It's a presentation app that takes your content or even just a prompt and creates presentations, documentation, or websites. I haven't really been a fan of specialized AI tools like this since a lot of the time, everything the specialized tools can do, you can do in ChatGPT or Claude. Gamma has changed my mind.
I have very little design skill, so a tool that can take over for me on that end is super valuable. I use Canva a lot for that, but Gamma goes a step further for me. With Canva, I still have to know colour schemes and stuff like that. There are some good templates for that, but it still requires me to pick what I think looks best, which is definitely not going to be what most people think looks best. Gamma helps with this because I can just give it a theme that I am going for and it will create a slide deck that matches that theme.
Gamma follows the freemium model and the paid plan offers decent basic features, a limited number of design credits and Gamma-branded exports. $10USD/month gives a bunch more credit and removes the Gamma branding.
AI News of the Week ๐ฐ
๐ฐ
Has generative AI hit a scaling wall? In other words, have improvements in LLMs slowed down/halted because training on more and more data doesn't lead to better LLMs? This could well be true, but even if it is, I don't see this stopping progress with AI because there are still other routes to improve LLM performance (e.g., let them "think" for longer like with OpenAIs o1-preview) or provide new interfaces (e.g., Claude's computer use)
A new math benchmark for AI that might actually hold up over time
AI companies (well, Anthropic is anyway) are starting to take seriously the possibility that AI systems will become conscious. This research firm believes that if AI can become conscious, then the moral thing to do. is to plan for that eventuality.
You may or may not know podcaster Lex Fridman. You may or may not like podcaster Lex Fridman, but somehow he is able to get really interesting people on his podcast. In this 5 hour ๐ฑ podcast (I've listened to about 1/3 of it so far) Amodei talks about the importance of AI safety and how Anthropic puts that at the center.
Askell is a philospoher by training and is now involved in developing Claude's personality and capability. She talks about the challenges of creating a system that is accurate and honest, asks follow-up questions without being annoying, and is not overly agreeable but sill culturally adaptable
I haven't gotten to the Chris Olah part of the interview yet, but he is an expert on mechanistic interpretability which maybe sounds kinda awful, but I find it fascinating. It focuses on trying to understand how generative AI actually creates its responses. It's kinda like being a neuroscientist, but for generative AI. One interesting project that Anthropic did this year was "Golden Gate Claude" where they basically made a version of Claude obsessed with the Golden Gate Bridge so that every response it generated was associated with the bridge somehow.
Watch on Youtube or listen wherever you get your podcasts (Lex Fridman episode 452)
AI Powered Pedagogy ๐ฉโ๐ซ
ChatGPT likes to tell a story about a girl names Elara who lives near a dense forest when you ask "Tell me a story" and this teacher used that to catch students using ChatGPT when they were not allowed. I don't propose using tricks to catch students cheating, but this story is interesting because it simultaneously shows the limitations of LLMs and the importance of good prompting.
Chegg (2022's favourite cheating platform) has lost 500,000 subscribers and 99% of its 2021 stock value since ChatGPT's launch
Up to 17% of peer reviews in computer-science journals are written by AI
See if you can solve these 10 GenAI Challenges from BCCampus. How to self-enroll
Upcoming Webinars (You may find these seminars interesting, but I don't have any insight into how good or informative they will be for you)
GenAI in Teaching and Learning 101 (UBC). Nov. 19, 2024 12-1.
Teaching and Learning with AI: A Sharing Conference for Educational Practitioners. May 28-30, 2024 in Orlando Fl.
The mAIn Event
Google's "Learn About" ๐
Following the success of NotebookLM, Google has launched Learn About, an AI powered educational resource that is supposed to enhance the learning experience by providing a structured and interactive format for learning about a topic.
Learn About is in the experimental phase and you can try it out by following the link below, but don't click on the link yet because you need to be in the US to use it. Or at least Google needs to think you are in the US, so you can just use a VPN to access it. The easy way that I do this is just use the Opera browser which has a built in VPN that you can point to the US.
How it works
Enter any topic (like "How is AI affecting higher education?")
Click through interactive lists to explore subtopics
Follow your curiosity without complex AI prompting
Get visual, organized content from reliable sources
The big picture
While other AI tools require expertise and careful prompting, Learn About aims to improve upon this by:
Handling complex AI interactions behind the scenes
Creating clear, structured learning paths
Reducing information overload
Making self-directed learning accessible to anyone with internet
Yes, but some limitations exist
Sessions can't be saved (yet)
Standard AI issues like bias remain
Google's history of discontinuing projects raises sustainability questions
What's next
While officially waitlisted, most Google account holders can access Learn About now (if you use a VPN to the US). Educators should test it as a potential tool to boost student agency and independent learning skills.
The bottom line
Learn About is aiming for the sweet spot between encyclopedia simplicity and text book depth, making it a promising tool for giving studentโs some autonomy in their learnign in higher education.
Reply