38 Comments

Kia ora from Aotearoa on the other side of the world, Alberto! Congratulations on reaching 1000... I have really enjoyed your writing during 2023, some of the most timely and insightful thinking about AI globally! I look forward to continuing to read in 2024!

My question to you: there seems to be no doubt now (to me at least) that we can actually perceive that AI is accelerating the advancement of more AI and human knowledge acquisition. 2023 was the year of multimodal / multimedia generative AI but I am starting to see signals that 2024's headlines will be more focused on AI pushing scientific and technological boundaries forward at rates we previously hadn't imagined - and as a result fields such as biotech, robotics, space exploration, brain-computer interfaces (and even potentially scalable solutions to climate change!) may be brought forward. From what you're seeing out there, who are the main movers to watch (outside of Deepmind, Xai etc) and what kind of headlines are you scanning for?

Expand full comment
Dec 6, 2023Liked by Alberto Romero

And congratulations on 1000 paid subscribers. :)

Expand full comment
Dec 6, 2023Liked by Alberto Romero

Curious to learn what you think are some of the biggest challenges that developers are facing while deploying LLM applications.

Expand full comment
author

That's a good question that I can't really answer as I don't do that kind of work!

Expand full comment
Dec 5, 2023Liked by Alberto Romero

Congratulations for this amazing milestone!

As a parent of 2 young kids and almost zero free time, I am really happy to get your newsletter in my inbox every few days, which gives me a very well put and balanced perspective of what is going on in this crazy AI world ;)

My questions would be

1) How do you manage to deal with so much information, perspectives, strong opinions.... And being able to condense them so nicely in an article? (Without going crazy in the process)

2) Which of the current AI tools / plugins would you say are influencing your work / personal life the most? (For better or worse)

3) Just a bit curious, which are your personal hobbies?

Expand full comment
author

Thanks Hector! I'm glad I can give you that kind of value :)

1) Great question! Very hard to answer without reflecting for longer but I'd say it's a mix of habit and method. Do it a lot and try to do it better over time. That's so generic as to almost be useless, but even I get lost in the details if I try to understand the exact factors that allow me to do this so much all the time!

2) I only use ChatGPT (with GPT-4) to create the images of my articles. I configured a GPT to absorb the content of an article and generate an image that best reflects what's written. It works pretty well, tbh, better than I'd expected. I don't do any writing with AI tools, though - and plan to never do it.

3) The things I do most that have nothing to do with AI or writing are singing (as an aficionado, nothing pro), working out, and reading (admittedly, that's rather related to writing lol, but I often read about stuff that's unrelated to my work so I think it counts as a hobby). Then, of course, going out with friends/gf, watching series, and playing video games. Pretty common stuff I have to say! Thank you for the question!

Expand full comment
Dec 4, 2023Liked by Alberto Romero

congrats on the milestone alberto!

i'm curious about your latest thinking on the value of learning prompt engineering for large language models. one of your recent articles said that openai killed prompt engineering with dall-e 3 and i'm wondering if this is likely to be the case with large language models anytime soon.

i guess my real question, should i invest my time and energy in developing my prompt crafting skills?

Expand full comment
author

Thanks Ivan! I wouldn't go too deep learning and memorizing all kinds of tips and tricks mostly because the best practices change every month. Just keep an eye on the space and develop high-level heuristics of what works best without much detail. I recommend you follow people like Ethan Mollick (here on Substack) who have a more hands-on approach to AI.

Expand full comment
Dec 4, 2023Liked by Alberto Romero

Congratulations, Alberto. Well deserved.

I’ve seen various AI thought leaders state that they believe some other type of approach different from the ubiquitous transformer is necessary to achieve AGI. What are your thoughts on that and do you have any favorites that you think may be good contenders to replace transformers? I realize that the public doesn’t really have a lot of visibility for these nascent approaches, but out of the ones you’ve heard about do any look more promising than others?

Expand full comment
author
Dec 9, 2023·edited Dec 9, 2023Author

Thank you John, and very good question! I don't have a clear preference right now but I'm keeping an eye on recent developments. I feel that the ground is shaking under the transformer as the hegemonic paradigm. Here's an example from yesterday: https://twitter.com/togethercompute/status/1733213267185762411, https://twitter.com/Teknium1/status/1733233296962953567 (as far as I know the problem they're targeting is the quadratic bottleneck of attention - the compute required to process a sequence of data grows quadratically with its length).

Was also thinking about the Mamba paper: https://twitter.com/_albertgu/status/1731727672286294400

Expand full comment

1) I wonder what you make of LeCun's recent comments regarding future AI development. He compares it with Orangutan society principles, which, according to him, do not have the dominance/hierarchy setup incorporated in other primate societies. He stresses the strict separation between the need to dominate/take over and intelligence (where the later does not imply the first). The argument is intended to take away the fear of AI taking over based on its innate intelligence alone. 2) Ok, as to selfish questions: I write the ShadowLands on substack, which is less than a year old. It brings links between traditional storytelling (stories by storytellers for audiences) and scientific discovery/the life of a scientist. Do you have any pointers on advertising this type of substack contribution to broader sources of potential readers? I currently use LinkedIn and Facebook only.

Expand full comment
author

Hey Michel, good to see you!

1) I agree with LeCun on that, but I don't think that's the main argument of AI doomers - not that a superintelligence would want to dominate us but that it would unintentionally and inadvertently kill us just because we're made up of matter and energy it could use for something else, i.e., it'd kill us as a means to achieve an instrumental goal for its ultimate goal. As far as I know, LeCun hasn't responded to that argument anywhere.

2) LinkedIn and Facebook are fine. Twitter/X is fine as long as you know how to avoid the link problem with the algorithm. I'd use Medium too (I do because I have 40K followers there so it's pretty useful). Notes on here is of course very valuable. I'd say it doesn't really matter which ones you use as long as you manage to learn how those work and optimize the funnels to your substack (e.g., the language you'd use on LinkedIn doesn't really work on Twitter and vice versa).

Expand full comment
Dec 4, 2023Liked by Alberto Romero

Cheers on your 1000 paid subs Alberto!! Genuinely! Well deserved.

My question is:

What do you think about action transformers, Fuyu-8B and the work/ambitions of adept.ai?

I think this tech has massive potential to be just as highly impactful as chatgpt.

Once the tool gets more refined and they extend it to work beyond just the web browser, people are possibly gonna be able to (for example) edit videos like a seasoned professional with simple natural language prompts..!

Expand full comment
author

Thank you, Simon!

I agree. AI capable of action (and planning) is the next milestone. I'm not sure about action *transformers* specifically because of the auto-regressive trap and the limitations of the transformer attention bottleneck (action requires many steps and confabulation is a real problem with that). But anyway, we will see this explored more thoroughly in the next couple of years for sure.

Expand full comment
Dec 4, 2023·edited Dec 4, 2023Liked by Alberto Romero

Hi Alberto,

Congrats on reaching this milestone - it's well deserved.

I've a question:

Given its hype, managers are sometimes tasked with exploring AI's feasibility within the organization. Do you have some suggestions as to what managers should be doing more, from a developer/technical person's perspective?

Thus far, I've approached it more as an exploration (rather than an exploitation) exercise, as the field is still new & isn't yet amenable to best practices & industry standards. This translates to incorporating confidence intervals into one's point estimates when coming up with milestones; using real options as opposed to IRR/NPV calculations when evaluating project feasibility; & maintaining skepticism when reading the news (In this regard, your substack is one of the few sources I trust), among other measures.

These measures mainly serve to manage/temper expectations up-stream (with upper-management) - the list of analytics/AI projects plagued by poor ROI is endless, even in "good" organizations.

However, this is just half the story. There's also the matter of down-stream management (of developers & technical personnel).

What might be your wishlist of things that you'd like managers of these projects to do more of?

Expand full comment
author

Thank you, Anna! I think you are absolutely correct here:

"Thus far, I've approached it more as an exploration (rather than an exploitation) exercise, as the field is still new & isn't yet amenable to best practices & industry standards. This translates to incorporating confidence intervals into one's point estimates when coming up with milestones; using real options as opposed to IRR/NPV calculations when evaluating project feasibility; & maintaining skepticism when reading the news (In this regard, your substack is one of the few sources I trust), among other measures."

I think the most valuable approach for you and your company is to learn and learn and learn without yet forming a strong opinion. Listen to those who know with skepticism as you say, because it's still super new.

I'd say you are doing all of this extremely well already given how you frame your situation and your question. I'm sure those above and below you appreciate that thoughtfulness greatly.

Expand full comment
Dec 5, 2023Liked by Alberto Romero

Your "...to learn and learn and learn without yet forming a strong opinion," is useful in so many areas of life besides technology.

Thanks, Alberto. I've enjoyed your insightful takes & look forward to more in the upcoming year.

Expand full comment

Congratulations Alberto!!! What a great milestone you’ve achieved. Yes, I will be seeking your advice and I appreciate your thoughtful approach to helping people out. Presently I’m attempting to distill the essence of a short book, but I would so like to present it as instalments on Substack so as to gain exposure. I’d like to be able to start to build towards a critical mass, but I’ve never really been much of a businessman. I suspect the strongest foundation is simply to write with as much quality as I can, and as consistently as I can. Although I’m consuming AI education every day and certainly have a vantage point or two, I don’t necessarily have the depth that you do. However, I have been actively assimilating factual information and points of view on this subject since the 1990s. Some of that has been strangely prescient and ahead of its time. I’m reflecting on an AGI Emergence story that I wrote in 1992. I suspect a powerful influence in that was the Stanislaw Lem story “Golem”, a story about earths first Super Intelligence (a short story that I highly recommend if you get a chance. It’s dense and prophetic, yet quite retro fun by today’s standards). I’ve been caught up in the subject ever since I was exposed to HAL in 1968.

When the time comes I would certainly love to run some material by you to get your sense of it. Right now I’m focused on the problem of alignment and how humans could significantly pose a problem as our species itself is so very often seriously out of alignment with itself. Hardly an admirable trait if we’re teaching AI by example. It’ll probably notice this staggering deficiency before long. Whenever that happens I sense that an AGI/ ASI would likely conclude that our juvenile species is actually too immature and very often acting irresponsibly, and will require serious “management”.

I really respect how you approach the subject and how are you craft your messages. I’ve always felt that my strength is in overview territory, as opposed to being down in the weeds. I always acquire some knowledge via osmosis when I’m down in the weeds, yet focusing on the weeds does not tend to play to my strengths, as I see it.

As I’m not immediately prepared with a good coherent question at this time I will ask that you expand in one of your template questions, namely to ask about the “three points” you alluded to. I realise my queries should lean into the highly pragmatic side of writing to gain a following. I have perhaps around 200 followers on Medium, and if I’m not mistaken last month I made about $1.28, so it’s probably clear I haven’t been serious enough about monetising my passion. To that end I certainly could use your advice. Thanks Alberto.

Expand full comment
author

Thank you, Paul. You nailed it here: "I suspect the strongest foundation is simply to write with as much quality as I can, and as consistently as I can." But as you may know, this is easier said than done, especially in such a generic way. Finding the specifics is the hard work each of us has to do!

Let me share the answer I gave to the same question (three points to grow the newsletter) elsewhere: 1) a) Develop a good habit but understand that you don't have to follow any particular schedule but one that works for you. Also, if you think it will take 5 months to achieve some milestone, assume it will take 10 instead. The number one reason why people fail to achieve things is because they stop too soon. b) Develop a good method to improve whatever you do while keeping the habit. The thing I did that I think most others didn't is nurturing an ambition to do things better and better and better (e.g., write better, research better, edit better, find better topics, do better marketing, etc.) c) keep doing a and b even when you feel it's not working. It's always working.

Expand full comment

Thanks Alberto! I appreciate your advice. Could you please speak to better marketing. I’m afraid I’m a bit of a bonehead in that department. Not my strength in any case.

Expand full comment

What are the three keys that allowed you to go from 0 to 1,000 paid subs?

Do you think new AI newsletters can still succeed if they start today? What’s the best approach?

Expand full comment
author

Hey Dave, I will copy my answer to David's questions below, which happen to be the exact same two!

1) a) Develop a good habit but understand that you don't have to follow any particular schedule but one that works for you. Also, if you think it will take 5 months to achieve some milestone, assume it will take 10 instead. The number one reason why people fail to achieve things is because they stop too soon. b) Develop a good method to improve whatever you do while keeping the habit. The thing I did that I think most others didn't is nurturing an ambition to do things better and better and better (e.g., write better, research better, edit better, find better topics, do better marketing, etc.) c) keep doing a and b even when you feel it's not working. It's always working.

2) I think they can (obviously not all will, most will fail). The key is to avoid doing the same thing as everyone else. Find a way to stand out that aligns with your idiosyncrasy and personality (so that you can keep at it for a long time, checking a) above) and do it well (checking b) above). In the beginning, it's okay to imitate (almost necessary) but keep an eye on ways to depart from others' way of doing things.

Expand full comment

Thanks--great points. A lot of people definitely don't stick with this kind of thing for the long term.

Expand full comment

Off Topic Time!! Woo-hoo!

I'd be interested in hearing more about the following, which is maybe related to AI?

It now seems beyond dispute that somebody unknown is flying hyper-high performance craft around our atmosphere. Whoever is in charge of such incredibly advanced craft may very well know a LOT about AI related topics.

Should we be interested?

Expand full comment
author

Hahaha have to confess I'm not surprised with your topic choice. My answer is: idk. But I read interesting analyses on Marginal Revolution by Tyler Cowen. I.e. https://marginalrevolution.com/marginalrevolution/2023/09/what-i-think.html and https://marginalrevolution.com/marginalrevolution/2023/08/my-views-on-the-ufo-hearings.html

Expand full comment

Thanks for links Alberto. I wasn't aware of Marginal Revolution. Yes, were you to write on the topic your piece would necessarily be highly speculative. Not the "do they exist" part, but clearly the "what are they" part.

=========

Topic #2: I just watched the Netflix film "Unknown: Killer Robots".

FULL FILM: https://www.netflix.com/title/81473681

TRAILER: https://www.youtube.com/watch?v=YsSzNOpr9cE

The film covers the military side of AI, and the topic seems very relevant to your blog. I don't recall you covering this much, so it might be new territory to explore.

Expand full comment
author

Yep, I would have wanted to cover military-focused AI much more but have so little expertise on the topic that would have required an amount of research impossible for me to do at the pace things were going. I now write faster and have many more ideas coming so I may find time to go deeper. It's interesting and high-stakes, much more than other things I write about!

Expand full comment

I'm not sure you need to become an expert on the matter, clearly an unreasonable hill to climb. The film above takes only an hour, there's a lot in there that can be commented on by civilians.

One thing I learned is that speed of decision making and deployment is crucial on a battlefield, and gives the fastest movers a huge advantage. Thus, the U.S. military is moving AI up the decision making chain to where it can have more sweeping control over an ever bigger picture.

Expand full comment
author
Dec 4, 2023·edited Dec 4, 2023Author

Not an expert at the level of people actually working on the military, but enough to teach it to others. Writing without knowing wouldn't have gotten me here! (A Netflix documentary is a good place to start but surely not sufficient, these things require much more research than that.)

Expand full comment

If I didn't have COVID, I could write about this all day long, and I know far less than you about AI. We don't really have to "teach it" but just generate enough interest that readers are encouraged to explore it on their own. Imho.

Also, did you know that the U.S. military now has full automated AI drones which at the push of a button will instantly zap anyone who tries to get you talk about UFOs? :-0

Expand full comment
Dec 3, 2023Liked by Alberto Romero

I like these two:

What are the three keys that allowed you to go from 0 to 1,000 paid subs?

Do you think new AI newsletters can still succeed if they start today? What’s the best approach?

And regarding Q*, look at this (which I happened to stumble across the other day):

https://twitter.com/HarperSCarroll/status/1729502499281764591?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Etweet

It's conjecture, but highly intelligent and extremely well informed and explained.

Expand full comment
author
Dec 4, 2023·edited Dec 4, 2023Author

Thank you, David.

1) a) Develop a good habit but understand that you don't have to follow any particular schedule but one that works for you. Also, if you think it will take 5 months to achieve some milestone, assume it will take 10 instead. The number one reason why people fail to achieve things is because they stop too soon. b) Develop a good method to improve whatever you do while keeping the habit. The thing I did that I think most others didn't is nurturing an ambition to do things better and better and better (e.g., write better, research better, edit better, find better topics, do better marketing, etc.) c) keep doing a and b even when you feel it's not working. It's always working.

2) I think they can (obviously not all will, most will fail). The key is to avoid doing the same thing as everyone else. Find a way to stand out that aligns with your idiosyncrasy and personality (so that you can keep at it for a long time, checking a) above) and do it well (checking b) above). In the beginning, it's okay to imitate (almost necessary) but keep an eye on ways to depart from others' way of doing things.

Expand full comment

Thanks, Alberto. It’s great advice. Funny how such a simple thing (get a little better each day in this aspect and that aspect and that one) can be so powerful. We hear that now from NFL players as a kind of mantra and it is somehow so easy to forget to apply to one’s own life.. For me I think that would be the primary discipline

Expand full comment
Dec 3, 2023Liked by Alberto Romero

And I'm proud to be one of those 1000 :) Congratulations Alberto!

Expand full comment
author

Thank you :))

Expand full comment