We should be happy this is the case. Great minds pursuing great aims is undoubtedly bullish for humanity. But how might we persuade these great minds to work on something which is not so positive?
AI is one such topic. Most well known AI researchers entertain some thoughts that an AI can be built which actively attempts to harm humanity. These beliefs have hampered AI research in recent years, most notably at Google where ethics committees and other “morality blockers” prevented Google from capitalizing on their breakthroughs in attention and transformers.
OpenAI was more clever with their pitch to concientious researchers. They said: “someone will build AGI, our aim is to get their first so we can control it”. They bolstered this pitch with a strong focus on alignment. You might say the Manhattan project, profiled in the movie Oppenheimer, had a similar pitch. Many scientists on the project were pacificists or had objections to the bomb, but they worked feverishly at it anyway so that the technology did not fall into the wrong hands.
Now, we are witnessing a backlash to this pitch, where OpenAI’s stance of keeping their model weights private (they have every right to do so) is arguably leading to more progress on the backs of open source models. Llama 34B, the second largest open source LLM at the time of this writing (after Llama 70B) has recently been tuned to run fast on an M2 Ultra and there is some speculation that adding a Mixture of Experts architecture to Llama could lead to GPT-4 levels of performance.
In this line of work, researchers are motivated by releasing AGI to everyone all at once, a third evolution from:
For ambitious projects, it’s a good idea to think through the core motivations of people working on them and be sure to tune the pitch to fulfill their deepest psychological objections to the work. On this front, we have to give Sam Altman and the leadership team at OpenAI credit for overcoming the objections of scientists circa 2017 and giving us all access to this great technology.
]]>Deeply understanding this limit is the key to using LLMs like ChatGPT effectively. You should not expect it to perform novel tasks, like writing a thought provoking blog or coming up with a new algorithm. Instead focus on tasks which you know others have likely performed before in some way.
Thus far, we have found ChatGPT to be exceptionally effective at the following tasks:
Write a Go program which uses the modules-aware packages package to search a codebase for types which have
functions with pointer receivers attached. Look for types where there are multiple variable names
for the pointer receivers. Use godoctor-rename to rename the variables using whichever variable name
is used in more functions associated with that type.
The unifying theme of these capabilities? There is nothing novel about completing them. That does not mean that the exact task has to be done before. But the ideas needed to complete these tasks are completely routine. Happily for us, most work is too.
]]>If you agree with Gordon, and many do, then the emergence of the powerful, mysterious, and subtle large language models (LLMs) such as chatGPT should be tremendously exciting. And it is not only chat interfaces but the last few years of AI have seen protein folding solved, full self driving deployed, and many other advances.
And yet, AI accelerationists claim that exponential increases in the capability of AI systems will result in massive job loss and re-alignment in the economy. Sam Altman recently asked “how much would a plumbing job cost versus building an iOS app in 2030?”, implying that the value of physical labor will sharply increase while AI will drive the value of digital labor to 0. Similar sentiments abound across Twitter and in conversations at tech parties.
If we have truly been in the midst of 50 years of stagnation, than we should be seeking advances which can re-acclerate us on a path of growth and innovation. AI could actually be that. Should we not embrace the chance to deploy scary AIs which operate in ways we do not understand and yet make everyone, from doctors to programmers to beurocrats more productive?
After decades of slow growth, surely a few years of technological disruption will be a good thing.
]]>Among the things that are working well on it now that were not when I first got it are:
When I first tried to control the lights in my home by stitching together Hue, HomeKit, the Watch, and Siri: it was a disaster. I ended up creating 8 shortcuts, 2 for each of the 4 Hue lights I had at the time. Each shortcut was either on or off. Then I added those shortcuts to the Watch face. This was so cumbersome that I gave up entirely.
Fast forward two years and I started from scratch setting up a new home. I collected all the lights into rooms and a prompt appeared to connect with lights with Homekit. Voila! I could now say ‘Turn off the Living Room lights’ to Siri. Each individual light had an on/off switch (with a dimmer!) in the Home app on the Watch. Even better, I could add new lights to any room and still turn them on and off with Siri.
Similarly, I recently received a prompt to use Apple Pay on the Watch in ‘Transit’ mode so I could wave my watch in front of the subway turnstile and buy a ticket. ‘Hey Siri’ works without any buttons. Sleep tracking now measures phases of sleep and resting heart rate in great detail. Activity and Health sends more useful notifications, especially downward trends in health. Notifications are grouped and stacked together and with a bigger screen on the watch I scroll through my notifications on it instead of my phone.
Besides praising Apple and myself for sticking with it to the point where I’m getting a great experience, these improvements highlight something I believe is unequivocally true: software is getting better.
Across all areas where I use software - from business apps to government portals to mobile devices to open source software - I believe the quality and user experience for software is steadily improving. This is great for entrepreneurs. It means that people’s expectations for what their software can do and how it should work will keep getting higher and higher.
This simplifies the task of building new products. Want to stand out amongst competitors? Pages should load quickly, information should update without reloading the page, and information hierarchy should go from simple to complex. Besides having one or two penetrating insights about how the product should work, entrepreneurs can focus on delivering a better experience of an existing workflow and gain traction. In my experience, this also dovetails with bottoms up adoption. A great UX will appeal to the actual users of the software who can then help win over the buyer from a competitor.
So for 2023, in the midst of pounding your head against the desk trying to edit a field in a huge enterprise app, take a second to appreciate how far we have come. Software is getting better, and better software means big opportunities.
]]>Most of the breakthroughs this year have been in generating new media. Dall-E, Stable Diffusion, and GPT-3 all generate text or images from prompts. The models are shockingly good at this task. The transformation is purely digital - someone types a prompt and the model produces a piece of content. If you believe that AI is mostly about the data then the progress here is unsurprising. The internet provides a nearly endless repository of content on which to train.
The second dimension takes sensor data from the physical world and maps it to a digital twin which can be further manipulated. Cashierless checkout, weather prediction, classifying radiology images, predicting machine failure, or any system which intakes physical data and uses AI to map it to something understandable by further software systems or humans fits this second dimension. Progress here has been steady but as you can imagine, training datasets are more limited.
The dimension with the slowest progress are closed-loop systems, or those which take in data from the physical world and then manipulate that world based on AI decisions. Self driving cars and robotics fit this third type. As we enter 15 years since the first Darpa competition it may be tempting to think that there have been self driving breakthroughs or that robotics has come a long way. But the reality is that it has been more of a grind. To date I am not aware of any robot which is commercially deployed to do things like clean a home. The story with cars looks better but a future where children never learn to drive still looks like a long way off.
When we think about investing or building in any of these dimensions, the areas with the most fundamental progress are the places where we can move to building applications and software interfaces to obtain widespread adoption. So it is viable to focus on building the perfect prompt input to generate copy for ads. Or build a saas service which writes blogs for companies. It would not make sense at this juncture to build applications for self driving cars - such as a car communication protocol to coordinate the most efficient commute in a city. Too much fundamental research is still to be done.
However that represents another opportunity. We know that teams which make research breakthroughs are very attractive acquisitions or they may be SPAC’d or see some liquidity event based only on the fundamental discoveries they have made. Lidar companies would fit this category. So it is important to know which dimension you are building for and where progress stands because this fundamentally impacts the go to market strategy for the company.
]]>Now, I think I have one. My favorite essay about startups is Paul Graham’s “Startups = Growth”. He makes the point that what truly defines a startup is truly ridiculous growth. If you stay in marketing long enough you will see months where some metric jumps 50%, or a quarter where the number of users for a particular product grows two orders of magnitude. The question for founders is - what if the whole company grew like this, consistently?
Growth is hard to get and hard to keep. If you are not in the act of conciously growing a startup, you may consider a marketing role at one to keep your muscles warm.
]]>Intelligence usually follows perception, or it is involved in entirely unique systems. Intelligence means predicting, learning, discovering, noticing. Intelligent systems may not be generally intelligent, but they are likely to become so. Copilot, GPT3, and Midjourney all exhibit signs of intelligence.
Data is the storage, transfer, availability, correctness, modeling, retrieval of pieces of information. Perceptive and intelligent systems consume enormous amounts of data. Many answers from data today are obtained manually. Databases, data warehouses, data lakes, data lakehouses, object stores, in-memory databases, client-side databases, and file systems store data. Data is streamed, subscribed to, queried, transformed, extracted, and updated.
Interested in these topics? Chat with us.
]]>