Bias Insurance in Artificial Intelligence

At Talla, we've had a lot of interesting overtures from large enterprises and have been thinking about the requirements to sell to them.  Big companies will almost always pay more to offset liability, and they expect even small startups to have good data access and data privacy processes in place.  This led to a lively discussion about A.I. algorithms and bias and whether we could be legally liable for any discriminatory biases in our algorithms.

Issues like this can sometimes slow the progress of innovation.  Part of the reason startups often best large companies isn't because large companies always miss the opportunities.  They often see them but can't go after them because they aren't willing to wade into murky legal territory.

This made me wonder if we will see a business model emerge where a third party testing house bears the risk of algorithm bias.  How would it work?  Say you are BigCo and  you want to put out a product that uses machine learning, but it isn't always 100% predictable as to what the outcome of using the product will be and you don't know exactly what data it may be exposed to once out in the wild.  You feel good that it won't end up like Tay but, all it takes is one person to sue you and say your algorithm is biased and the odds you have to settle are pretty high.  So you turn to a third party who indemnifies you.

This third party would be AlgorithmCertCo, and for a very high initial fee, and an ongoing yearly re-evaluation fee, they will certify your app to be free from bias.  They would run tests on specially compiled data sets that would provide "proof" that your algorithms don't provide whatever kind of bias might be legally harmful for your potential use case.  Thing of it like a credit rating for algorithms.

There is currently a small field of machine learning called adversarial machine learning,  which looks at ways in which to hack/break/fool neural networks and other learning algorithms.  It is an easy step to think about this for use cases of breaking neural nets in ways that show they are biased across something like gender, race, or sexual orientation.  Now if an algorithm passes these tests, BigCo can stamp it "bias free" and, if someone sues for bias, AlgorithmCertCo is on the hook.  And companies will pay lots of money for that.

The Three Levels of Competition in Personal Assistants: Where Google, Amazon, and Apple Are Fighting

With Apple's World Wide Developer Conference coming up Monday, I've been hearing a lot about how Apple is behind the game on artificial intelligence.  Between the Viv demo, and the success of the Amazon Echo, its easy to see why people are starting to count out Siri.  It doesn't help that Apple's stock price is down too, and that negative halo effect is causing perception problems with all of Apple's business lines.

To understand who might win this space and why, you have to understand that these personal digital assistants operate on three different levels of core technology:  voice, intent, and implementation.  Solving the voice level is a signal processing problem.  Success at this level means understanding various dialects and conflicting words (did you say "socks" or "sacks"?).  Amazon Echo, from what I've read, is currently the best at this level, surpassing even humans.

The next level is taking the sounds the device heard and converting them to an intent.  If a device hears "show me my pics", am I asking about pictures, or did I mean "show me my picks" because I was talking about my fantasy sports draft?  Taking language and understanding the intent is perhaps the most difficult level, because of the ambiguity in human language.  I don't know for a fact that Google is the best at this but, they should be.  Google probably has the best data set in the world for mapping natural language to intent, so they should win this level.

The final level is task/knowledge oriented.  Once an assistant understands my intent, can it answer the question or execute the task?  This is really a function of how many things the assistant can do, and with Apple opening Siri to developers, I expect Apple will win this level.  What I mean is, Apple has the most robust developer ecosystem for developing end-user functionality on their platform, so my guess is that, a year from now, Siri does the most stuff.

The question then becomes, which level is most important to win?  I'd argue that the top two levels, the ones currently won by Google and Amazon, are the most likely to be commoditized.  If I'm right, and Apple wins at the level that is most important - the number of things an assistant can do - then Apple is fine going forward, despite what some people are saying.

Analogy Making In MicroDomains: The Next Wave of A.I.

For much of the last decade, the hot space in A.I. was machine vision.  The release of ImageNet gave researchers a common target to work on, and a very large data set, and spurred dramatic advancements in machine vision accuracy as a result.

As machine vision for common problems started to reach human levels of performance, many of the best and brightest in the field realized that the marginal contributions to machine vision were low, and turned to other areas.  Natural language understanding started receiving a lot of attention.  A.I. related work on natural language has been an explosive space and has moved forward rapidly over the past few years.  I suspect that in another 3 years we will have made remarkable progress as an industry.

In parallel, some fields that are lumped together with A.I., like predictive analytics, have moved forward pretty rapidly as well.  With all the investment in these spaces, they are moving forward at a nice pace, which raises the question... what is next?  A.I. technologies rise and fall in popularity.  Neural approaches are peaking and will probably start becoming less popular by late 2017.  Bayesian approaches are getting a fresh look and starting to ramp up again.  Genetic algorithms are stale, and I think will remain so for a while.  So what technology will be the next to start rising in popularity again?

My guess is A.I. analogy makers will start to rise again in 2017.  Many of you probably read Doug Hofstadter's "Godel, Escher, Bach:  An Eternal Golden Braid."  If you haven't, it's an amazing book so check it out.  Hofstadter was an early champion of analogy making as intelligence.  The reason analogy making is primed to rise is that, neural networks have primarily improved our accuracy on classification.  Classification is at the bottom of the intelligence hierarchy and so, it makes sense to start climbing that hierarchy again and perpetuating our advancements upward.  We can build on the better classifiers we have to do better analysis, reasoning, and analogy making.

This time around, analogy making will start by focusing on microdomains.  Just as neural networks made dramatic progress by focusing on particular data sets that were conceptually narrow, analogy making will make progress by focusing on microdomain analogy making, where success will be easier to classify and results will be more useful.

The best thing I ever read about analogy making in microdomains is Robert French's "Subtlety of Sameness", which is French's PhD dissertation turned into a book.  French created a program called Tabletop.  The way it worked is, a table is set a certain way on each end.  French then points to an object on his end of the table, and the program has to point to "the most similar object."  When the table ends are set identically, say, both with a plate, a fork on the left and a spoon on the right, this is easy.  Touch the spoon and the computer will touch the spoon as well.  But what happens when the table ends are setup differently?  What if there is no spoon?  In that case, the program may touch the item in a similar location on the table.  But what if that item is of a different class?  Maybe it's a glass rather than a utensil.  In that case, the computer may touch the closest utensil.

Tabletop explores analogy making in interesting ways.  Something could be similar in substance, or in a similar class, or in a similar location, or have a similar function.  The algorithm works both tops down and bottoms up, which is an approach I think will become more common as A.I. moves away from neural network approaches into other areas.  (Or maybe applies bottoms up and tops down NNs to analogy making somehow?)

Analogy making is the next step up from classification and prediction, and will enable us to solve a new level of problem in A.I.  On top of that, advances in lower levels of A.I. technology haven't yet pushed analogy making forward, but I think they will.  That's why I expect it to be fertile ground for innovation starting soon.

I think 2017 will see the early seeds of analogy making rising again on the A.I. front.  If you want to be part of that rise, or bet on it as an investor, it makes sense to start refreshing yourself now.  Reading about Tabletop is a great place to start.

Cognitive Ergonomics And Chatbots

At the Xconomy Robomadness event in Boston this past week, I heard former Evernote CEO Phil Libin speak on a panel about business use cases of A.I.  I've talked to many VCs about conversational interfaces and I think that at the moment, Phil has thought more about them than any other VC I've met.  During the panel, Phil use a term that I hadn't heard before, but really liked - "cognitive ergonomics."   A quick search turned up quite a few articles but, it is clear people haven't thought about this concept in terms of Natural Language Experiences (NLX).  I'd like to take this post to outline a few key ideas about that.

A phrase from the Wikipedia definition sums it up nicely.  "Cognitive ergonomics studies cognition in work and operational settings, in order to optimize human well-being and system performance."  From a NLX perspective, this means a "bot" has to be able to communicate in a way that minimizes the cognitive load on the user.  Since design is generally all about tradeoffs, I have been asking myself "what is the tradeoff when you minimize cognitive load?"  I think it is time.  

As the CEO of a company that is building an intelligent assistant, I spend a lot of time in team discussions about how Talla should communicate.  What I'd like to do is share a few cognitive ergonomic ideas we are talking about.  Feedback is of course welcome if you have input on these ideas.

1.  Cognitive Efficiency - This is a measure of how much cognitive effort is saved through the interface.  For example, compare 3 alternatives to getting calendar information for something 2 weeks away.  In the first, a user has to exit a messaging platform, open a calendar, scroll to the right day, and find the event on the calendar.  In the second example, a user can ask Talla, right from a messaging platform, a question about the day, or the event, thus saving several steps.  In the third example, Talla can anticipate that the user may need that information based on ambient information, and volunteer it without the user asking.  These are three steps of increasing cognitive efficiency.  

2.  Cognitive Return on Investment - This has to do with the ratio between Cognitive Efficiency of a new task, and the investment required to teach Talla that task.  Tasks that are easy to teach but super powerful have a higher ROI that tasks that are either difficult to teach, or, easy to teach but limited value.

3.  Cognitive Fatigue - Some people are just draining to interact with.  Others are very encouraging and engaging.  Bots could have the same types of impacts and so should attempt to minimize cognitive fatigue.  Good bot interactions should have the same impact as having a helpful conversation with a human - not too much info beyond what you need, concise and direct, and helpful.  

4.  Cognitive Fit -  Just like everyone has a favorite style of chair, everyone has a favorite style of conversation.  Bots should match their language style to the kind you like by learning your interactions.

Those are some initial ideas on what cognitive ergonomics for conversational interfaces may look like, but I'd love to hear your ideas.  Please email me with your thoughts, and if it makes sense in the future I'll do a follow up post with a deeper list of ideas.  

How Machines Will Run Our Lives Without Becoming Sentient

Talla's funding round included some participation from an AngelList syndicate.  While trying to finalize things, I sent an email to someone at angel.com instead of angel.co.  It came back so I tried to resend it but, no matter what I did, Google kept changing the .co to .com.  It did this because it's "smart."  I had sent 1 email to angel.com and zero emails to angel.co, so, surely this was a mistake and I didn't really mean angel.co... right?  It took me a couple minutes to try various combinations of entering and backspacing/escaping/tabbing to figure out how to get Google to stop "correcting" angel.co to angel.com.  

Much has been written about the concern of sentient machines terminating us all in some sort of hostile act.  And much has been written about why sentient machines will be nice instead of deadly.  I want to make a different point, which is - the machines may take over long before they are sentient.  Let me give you an example.

I'm sure someone will build a machine learning tool that helps suggest what you major in when you go to college.  Some people will follow the advice of the machine, and some won't.  Those who do follow the advice of the machine will probably end up happier with their career paths, because the machine will be effective at some level and, there may be a placebo effect component to it as well.  Over time, this will make the machine seem even better at placing high students into college majors.  

Everyone will use the machine without question.  And the machine can't make a "bad" recommendation intentionally.  The machine can't experiment with your life in order to test its algorithms.  That would be immoral.  It can't say "my algorithms suggest you should study Accounting but I'm going to suggest Graphic Design and see what happens."  

So, over time, it won't just be college majors, but career paths, job changes, employers, skills.  The machines will suggest everything, and we will take their suggestions.  They will make the decisions, not us.  At that point, the machines don't even have to be sentient, but will direct most areas of our lives.  In the end, the machines win wether generalized A.I. is built or not.  The machines win because we humans will do anything to avoid making the "wrong" choice.  The machines don't have to take over, we are already in the process of willingly turninging our lives over to them.

The Coming A.I. Botageddon

Everyone is building a bot.  At Talla, we talk about this a lot because we are too, so we have to make some bets on how this space plays out.  Around the office we frequently use the term "Botageddon" because we are constantly discussing what happens when you have dozens of bots on your messaging platforms that are all trying to do various tasks for you.  It's too many and it can create a terrible user experience as a result.

The underlying cause of this bot problem is that, the state of A.I. is not good enough to build a good general intelligence bot.  Some big companies are trying, and you've probably used their products (Siri, Cortana, Alexa), and come away amused but not impressed.  Some startups are trying too,  but I can't imagine they can outwit these big tech companies on something like this.  The result is, a lot of companies (mine included) are taking approaches to building vertically targeted bots.  By limiting the scope of what the bot has to know and do, it is possible to build something pretty good for that domain.  And we all know that if we can get something good enough to sell, we can keep improving it until the technology one day allows us to build a real human like metamind bot.

With all these vertically targeted bots, how many will an average user need?  I really doubt that you want to use a banking bot, a fitness bot, a dating bot, a restaurant bot, etc, etc, etc.  Since the current trend is for all the bots have names, you will forget which bot does what if you have too many.  And sometimes the bots may conflict with each other.  The big question then is, if the user experience of using 47 different bots isn't a good one, how does it get resolved?  Where does the ecosystem settle in to a stable equilibrium?

I see 3 key possibilities for resolving this Botageddon.

1.  Industry Consolidation - I think this scenario plays out if the technology and algorithms improve fast enough that we move quickly from a period of too many bots to a period where building a generalized intelligence bot is feasible.  In this case, the initial winning platforms/bots buy up or merge with the smaller niche bots that didn't achieve scale.  Some players get to a general intelligence bot by combining lots of niche bots.

2.  MetaBots - In this scenario, meta-bots emerge that manage and route requests through the other bots.  This could take two forms.  One form is meta-bots that are intentionally designed to be meta-bots.  Much the way Google is a search engine that helps you find the right information on the web, a meta-bot could help you find the right bot to do the task, or get the information, you need.  The second way could be less direct, with some popular bots becoming sort of like meta-bots by default, maybe because they are the first to build bot APIs for easy integration, and because users are most comfortable with their interfaces.

3.  Platform Management - In this scenario, the messaging platforms on which most of these bots are built will serve the function of routing requests to the right bot.  So, a Slack or WhatsApp or even a mobile platform like Android might develop some functionality to do the routing.

As the bot industry emerges, there may be other solutions that emerge, but for now, these are the 3 ways I see the bot industry playing out as it matures.  If you have other ideas, I would love to hear your feedback.

An Interview With Erik Mueller, Co-Creator of Watson

Erik Mueller was one of the original engineers behind IBM's Watson, who left last year to go out to start his own consulting company.  I had the chance to ask Erik some questions about his views on A.I. related topics.  The interview is published below.

1.  You were part of the original Watson team - how did that project come together and what was your role on the project?

I joined the group working on Watson on February 12, 2010, a year before Watson beat the human Jeopardy champions. I contributed the UnitQuestionAnswerScorer and StoryProblemDetector components, helped improve the ESG parsing, and helped with LAT detection. In addition to helping Watson win Jeopardy, I was one of three researchers in charge of getting Watson ready for its application to medicine—the others were Tony Levas and Sugato Bagchi. After Jeopardy, several members of the group joined Tony, Sugato, and me to continue work on healthcare. We created two systems. Watson for Healthcare is described in our AI Journal article. My main contribution to this was the SKB structured medical knowledge base, which was mined from medical texts. WatsonPaths is described in an IBM technical report. I created custom models for specific types of medical subquestions, like going from a disease to a finding or the reverse.

2.  What was the most difficult problem to solve when building Watson?

There wasn’t any single difficult problem to solve when building Watson. Once the basic architecture was in place, the main problem was driving up its performance to the desired level. This involved a repeated process of finding and fixing errors that the system or components of the system were making.

3.  Deep learning is all the rage in A.I., but Watson is more of a "cognitive computing" approach.  How would you explain such an approach to an audience of business executives?

I would urge them to be open-minded in considering both approaches and how they can be combined. Watson exploits a large body of documents, whereas deep learning exploits a large number of training examples. (Watson uses a smaller number of training examples to learn to rank answers.) Deep learning can be used to build better components within Watson.

4.  When you and I last spoke, we talked about how humans reason symbolically.  Yann LeCun at Facebook has implied that "thought vectors" will basically solve AI.  You seem to be one of the few skeptics here.  What do you think an eventual solution to general A.I. looks like, technically?  Will it be deep learning?  Symbolic?  Or even something that we haven't solved yet?

That’s interesting. He may be right. I’m starting to think that there’s less difference between symbolic AI and neural nets than I originally thought. In my new book, Transparent Computers, I talk about the fact that we can implement reasoning—and the explanation capability that comes along with reasoning—a number of different ways. We can use symbolic inference rules to generate each step in a reasoning chain, and we can also use a neural network to do this. I think we’re still at the early stages of working all this out. There’s a lot that needs to be invented.

5.  If you were going to start a PhD today, and were interested in A.I., what problem or technology would you choose to focus?  

Right now, computers are ubiquitous, but they’re getting more and more annoying. I’m very interested in how we create intelligent systems that we can understand and that are transparent.

6.  I wrote a post about OpenAI being a dangerous thing, actually causing the problem it is trying to solve.  Do you agree or disagree?  

I think your analysis is dead on.

7.  Do you have any concerns about technology and the future of A.I.?

I’m keeping an eye on it. There have been some amazing results recently that seem to indicate AI may be progressing faster than we thought it would. On the other hand, the problem of narrative understanding is as far from being solved today as it ever was. All you have to do is take three sentences at random from a newspaper and think about what it would take for a computer to understand them. The training data needed to get a neural net to understand those three sentences is fairly large. Think about the training data needed to understand arbitrary sequences of sentences. I don’t think researchers have an appreciation for how large the space is.

8.  What is the most interesting advancement/research you've seen in A.I. in the past 12 months?

I’m impressed with the work at Facebook on using memory networks to understand narratives. It’s amazing that this works at all. But the narratives are very simple and a far cry from naturally-occurring text.

9.  What business opportunities do see for cognitive computing to have the most impact?  Any areas where it isn't being applied that entrepreneurs should be considering?

There are lots of applications in healthcare. One area that I'm very interested in is increasing transparency in medicine. When you or your loved ones are in the hospital, it's hard to understand what's going on. You need to be able to see exactly where you are in diagnosis and treatment protocols using easy-to-understand diagrams. Domain-specific Siri-like intelligent assistants is another big area.

10.  You left IBM last year to consult on your own, what kinds of work are you doing and what types of clients are you accepting?

I help clients use cognitive computing to solve a range of business problems. I’m seeing the most activity in the following areas: automotive applications, healthcare, Twitter analytics, entertainment, sports, advertising, data analytics, insight, quantitative trading, query expansion, education, tutoring systems, conversational agents, and highly-focused intelligent assistants. I really like it because there’s so much variety.

Will OpenAI Inadvertently Create The Problem It Is Trying To Solve?

By now you have probably read about OpenAI, the project started by Elon Musk and Sam Altman.  The purpose of OpenAI is to do top notch AI research and open source it.  The idea behind the group is that, if anyone and everyone has access to cutting edge AI technology, then creating a killer AI is difficult because there will be a plethora of AIs to balance out any bad AI.  Before this group existed, I wasn't worried about a killer AI.  Now I am.  

To explain why, let me start with an analogy.  Nuclear bomb technology is dangerous and destructive.  Should we open source nuclear technology and make it easy for everyone to build a nuclear bomb, under the idea that if a few bad guys have nukes but more good guys do, then we are safe?  Now admittedly, AI technology is different than nuclear bomb technology, so I want to highlight a few examples of where they are different, and show why I believe AI technology is even more dangerous to open source.

One of the interesting things about AI, that was highlighted in both Nick Bostrom's book and Roman Yampolskiy's book on the dangers of future AI, is that they both point out that we don't know exactly what the "takeoff" of an AI looks like.  If there are accelerating returns to intelligence, which most people in the field believe there are, then it is possible that creating the first super AI, just by a few hours (or even minutes!) could set off a series of events that put that AI on top of the world forever.  Why?  Because the first AI to start to improve itself, with just a few extra hours of learning and improving (machines can do A LOT in a few hours) could build a gap that now no further AI could ever close.  By getting smarter faster, and having a small head start, it could also figure out how to stop, delay, and damage any competing AIs.  

If this scenario isn't true, and AIs don't have a rapid accelerating takeoff, then the things OpenAI worries about aren't things we need to worry about.  Given a bit of time and resources, smart people will quickly figure out how to catch a non-accelerating AI.  So, OpenAI only needs to worry if there is an accelerating AI.  Let's work from that assumption to show three reasons why OpenAI is a problem.

1.  At the moment, we don't know where the key breakthroughs in AI technology will come from.  By putting top tier AI technology in the hands of anybody, we increase the chances that it gets in the hands of someone careless, or worse, nefarious.  Hackers with bad motives will now have access to tools and techniques that would have otherwise been off limits to them.

I believe Musk and Altman would probably counter that, since we don't know where the key AI breakthrough will come from, its all the more important that we open source this stuff and encourage collaboration.  But I believe that in general, there is a correlation between intelligence and wisdom, and if it takes an IQ in the top 1% to solve AI, the odds of someone in that group being wise about the uses and safety are more likely than if everyone in the top 20% of IQs gets to play around with it.

2.  The odds are unlikely that OpenAI will pursue the right technologies, thus making the group irrelevant.  If you don't participate deeply in the A.I. community, you probably don't realize the vast number of approaches to generalized intelligence that exist, and how different they are.  The Convolutional Deep Learning neural net approach is hot right now, but Probabilistic Programming and other Bayesian approaches are threatening to dethrone deep learning.  Plus you have guys like Doug Hofstaedter still working on analogy based approaches, Watson and IBM's cognitive computing and symbolic logic approaches, and Numenta's HTM approach. On top of that you have hardware based approaches like spiking neuron chips and neuromorphic engineering.  I don't see how OpenAI can possibly stay on top of all of these.  

You could argue though that surely a group with the backing and reputation of OpenAI will have good access to all these ideas and therefore can quickly come up to speed on a new technology if it proves promising and looks like it might be the key breakthrough, right?  Well, keep in mind that convolutional neural networks, the current hot fad in AI, were laughed at for two decades and had very few people working on them until Yann LeCunn and team exploded on the scene and shockingly beat the pants off every other image classification approach a few years ago.  It's highly possible that the key breakthrough idea in AI ends up being something that was ignored for many years before it has its breakthrough moment.

3.  The OpenAI approach assumes companies like Google and Facebook, who are on top of the AI world, will be careless or evil if they create an AI.  But will they?  I personally feel like if Mark Zuckerberg or Larry Page was in control of the world's first superintelligence, that we are probably in a pretty safe spot.  Those are two people who have no incentive to destroy the world, and have made enough money that their operational motives at this point are less financial and more ideological.  I think they would both be very thoughtful about AI development.

For me then, the status quo of having most of the AI brainpower concentrated in academia, Google, Facebook, and a few other companies, is actually very comforting as I think about possible negative AI consequences.  And OpenAI, on the other hand, scares the shit out of me.  

But when I back up and think about it, Musk and Altman are probably much smarter about this stuff than I am, so surely everything I have just mentioned has already occurred to them.  So why did they create OpenAI?  In my opinion, its because they don't have a major stake in Google or Facebook, and they want a way to get an economic stake in whatever comes next.   It's a way to weaken those companies and get their own piece of the upside of AI economics.  Much the way that I use this blog to selfishly generate leads for angel investing in AI, I believe Musk and Altman are just doing this to increase their chances of being part of the next deca-billion dollar tech company.  They aren't worried about AI taking over the world, they are just worried they will miss out when that step function breakthrough occurs. 

The Beginning of the End of Black Box Machine Learning

A research paper came out recently called "Convergent Learning:  Do Different Neural Networks Learn The Same Representations?"  This paper is an important one that hasn't been discussed much publicly, so I want to take this space to talk, in layman's terms, about why this paper matters and the impact it will have on machine learning research going forward.

First, let me explain three things about neural networks for those of you who aren't technical.  One is that the weights of neurons are usually initialized randomly, so two neural networks trained on the same data set never start from the same initial state.  The second is that neural networks learn what are called "representations."  One way to think of a representation is to think of the key features a neural net uses to classify something.  A neural net training on a character by character language set might learn, for example, that "q" is almost always followed by a "u" in English.  The third thing to note is that the middle layers of a neural network are called "hidden layers" in part because we don't know how they work.  Neural nets tend to be black boxes inside.  

That last point is important and ties to the core message of this paper.  It isn't that we are incapable of understanding the inner workings of neural nets, its just that doing so is difficult and doesn't really impact the problem we are trying to solve.  If the neural net reliably solves our problem, do we care how it works?  This paper is arguing that yes, as the use of neural networks propagates, it is important to understand how they work.  So, the research that generated this paper looked at identical neural nets, trained on identical data sets, to see if their internal states ended up similar.  

I won't go into the details of how they performed the analysis.  You can read the paper if you are technical enough to follow that.  What I will point out is that the key finding was interesting.  Different neural nets do tend to learn some of the same features for certain data sets, even if the way those features are represented isn't identical between the neural nets.  But, there are also many features learned by one neural net that aren't learned by other neural nets.  So it appears that, for common classification problems, there are a few salient features that every trained network will learn, but the full set of features varies slightly.

This work is important because starting to understand how neural networks work inside will undoubtedly spark more ideas and innovation in the space.  If you are technical, you should read the paper.  If you aren't, you should follow this conceptually as more research takes place.  It will be an important area.

Machine Learning As a Fashion Industry

This week, Yann LeCun came to speak at M.I.T.  His talk was great and, like LeCun, I am a big fan of both deep learning and the vector based embeddings that have become popular in ML.  The ability to process math, instead of logic, is indeed powerful.  But one thing disturbed me about the talk.

During the Q&A session, LeCun was asked several questions about other technologies, including the Hierarchical Temporal Memory Approach used by companies like Numenta and Cortical.io,  and spiking neuron approaches like those used by IBM's Truenorth chip.  LeCun's criticism was that neither approach had done well on standard data sets used by the machine learning community, and had not been able to beat any existing benchmarks.  This would be a good point were it not coming from a guy who admittedly toiled away on Convolutional Neural Networks for two decades, while they (the CNNs) were unable to beat any existing machine learning benchmarks, and people thought he was wasting his time.

It bothers me because I've seen too many technology waves come and go, and deep learning, while powerful, feels like a fad.  People seem to think its the solution to every problem, but it reminds me of AJAX, user generated content, noSQL, containers, and other technologies that were indeed interesting, but not panaceas.  What surprises me is the lemming like nature of machine learning research - with both academia and industry taking a bandwagon approach, and criticizing those who don't jump on the bandwagon.

One of the things I admire about LeCun is his conviction in his approach, and his long term view.  I hope his attitude doesn't discourage other researchers who share those rare traits.

Will Intelligent Machines Market Themselves?

Today, when you sign up for a demo or free trial of a software product, you are usually put on an email drip campaign.  Every few days, you will receive an email that the product owner believes will help you understand the product better and hopefully buy.  But what happens when your software is an A.I.?  Will you need a drip campaign?

Think about the way you buy a phone, or an appliance.  You do some research online, you look for reviews, you go to the store and talk to a salesperson.  But as appliances are smart, or, as robots become a major purchase, how will we buy them?  The interesting thing about the intelligence revolution is that, as intelligence gets built into everything, those everythings can sell themselves to us.  The question is, will they?  I'll take a quick look at both the "yes" and "no" arguments for the issue.

Yes.  Machines Will Sell Us On Them - Most people don't like being sold to, and that is part of what makes sales so challenging.  As we start to buy products that have a brain, it will feel less sales-y to talk to these smart products.  By asking them questions about themselves, and allowing them to ask us questions, we will feel like we are building a relationship, not getting sold.  Products that educate us about their benefits will make us feel more comfortable, and the products themselves will probably learn what to say to make sure we feel good about our purchase.  So yes, intelligent systems will do their own selling.

No.  Machines Won't Sell Themselves - Labor is currently the only thing we buy that has the capability to sell itself - human labor that is.  And how do we approach that?  There are really two markets - human service providers selling their services to us, and job candidates selling themselves to employers.  In both cases, my personal observation is that the sales aspects of those relationships are more muted than the sales aspects of inanimate objects.  People, when selling themselves, try to come off softer, less cheesy, and less like they are selling.  So, as machine intelligences approach and surpass humans in capabilities, we will see them more like humans, and expect them not to sell us on them too hard.

I don't know which path is more likely to happen, but we are in the early stages of A.I.s that can talk to their owners, and I expect to see a lot of marketing and sales experimentation within companies building those A.I.s.  Or maybe in the end, someone builds a negotiating A.I. that we can all buy and just avoid the sales process altogether.   

If you like this post, and are interested in machine intelligence, sign up for my newsletter and get a curated list of links and papers.

Machine Learning For Small Data

Machine learning has risen in part because of the rise of big data.  With more data available, techniques like unsupervised deep learning have been able to make tremendous progress.  But what about small data sets?  There are many more small data sets in the world than there are large data sets.  And humans seem particularly good at learning from small data sets.  In fact, large data sets overwhelm us.  So the question I want to briefly address in this post is:  Will there be a movement of machine learning for small data, and if so, what can we expect from it?

I had this conversation recently with the former head of machine learning for Youtube, who pointed out that part of the lack of progress in this area is that the companies who do most of the practical machine learning research don't have any incentive to investigate small data machine learning.  Their large data sets are an advantage, and of course good business is always to play to your strengths.  The only thing Google, Facebook, and Amazon gain from solving some small data problems is a bunch of competitors.

I recently got a new Android phone with a great fingerprint unlock function.  To activate it, I had to "train" the program by repeatedly applying my finger to the sensor.  The program actively encouraged me to move it slightly left, right, up, and down on each press.  I don't know the algorithm behind this process but I imagine it was some variation of a machine learning algorithm.  The interesting thing is, it was learning from a single data set, unique to me, that I was quickly generating.  The whole process took less than one minute.

While not much has been written about machine learning for small data, I can imagine cases where it is useful - most of them related to the activity of a single user.  For example, if the goal of a program is to anticipate my morning routine, it should be able to learn after just a few days of observing my routine.  It doesn't need 1000 days.  It doesn't need the morning routines of others.  But, such a program would also not have a very good way to predict those rare times when I deviate quite a lot from that routine, and the reasons for my deviations.

So what can we expect for small data machine learning?  I expect it to arise from the areas of A.I. that interact more with humans.  Such products will most likely be targeted to a single human user, and therefore, will need to learn more about that human owner.  The data set of owner actions, which will be small by the standards of machine learning data sets, will actually be the most useful data set for the job.  The impact this will have on machine learning algorithms is a move away from probabilistic methods and a move towards more situational specificity.  We will see algorithms that are basically collections of specific rules, that use decision tree like structures, but are very adaptive.

With small data, opportunities for automatic feature engineering from tools like deep learning won't be available, so what will?  I believe we will see a resurgence in the old idea of analogy making as key to learning.  Symbolic ontologies that can deal with multiple data types will be used to apply some guesses to what kind of structure might work for a given small data set.

I haven't seen much work done in the space so, if you have ideas, or have seen projects or papers based on small data set machine learning, please send them my way.

Flywheel Metrics: The Economics of Machine Learning

I write some things in my newsletter that I don't always publish here on the blog.  My last newsletter contained a brief explanation of why we are about to see machine learning bubble 1.0, but why it is entirely rational.  My framework for thinking about this comes from a book I read this year (which I highly highly highly recommend) called Technological Revolutions and Financial Capital.  The gist of why there will be a bubble is that, it's a relatively safe bet that some massive companies will come out of the machine intelligence space.  My guess is that you get a few dozen billion dollar companies, 5-10 decabillion dollar companies, and one centabillion dollar company.  That distribution seems to match what has happened in other tech revolutions.  But, it is very very hard for people immersed in this space to pick which companies will end up that way, and it's even harder for investors who don't understand the technology or applications very well.  In that scenario, it's entirely rational for a venture fund to throw money at machine learning startups hoping they hit one the decabillion dollar winners.  After all, VC funds aren't mutual funds.  The specific purpose of the money they raise is to take massive risky bets that have huge payoffs if they are right.  Of course, that type of behavior causes a bubble, but the bubble was, from each individual VC's perspective, entirely rational behavior.  

As the bubble rises, pops, and the real companies survive, an economics of machine learning companies will be developed.  I've thought about this quite a bit and decided to take a stab at what the important metrics for a machine learning company might be.

I want to qualify this post by stating that many machine learning companies will just be Application X with machine learning added.  In those cases, the economics of those companies will be dictated more by the Application X space than by the machine learning space.  I want to focus on new companies - companies building new applications that weren't possible before machine learning exploded.

This post will be long enough without considering revenue, so lets look at the cost side for today, and what is different from traditional companies.  

New industries like this sometimes bear slightly higher infrastructure costs and personnel costs, as infrastructure isn't optimized for the new use cases, and qualified personnel are hard to find which drives up compensation.  But both of those trends typically right themselves over time, and the deltas from normal companies are bearable.  So what is different that we have to be worried about?

I think there are two key issues:  data sets, and human intervention.  Initially, machine learning companies will take the low hanging fruit of existing data sets, but over time, entrepreneurs will realize that, if they could just buy/build a dataset about X, they could apply machine learning and have a really valuable product.  That could be really expensive.  One of the key skills of the best machine learning entrepreneurs will be the ability to get creative about how to get data.  Look for companies to track a metric that is something like Cost Per New Data Point.  That metric will be compared to Value Per New Data Point.  In other words, does the product improve enough from a new piece of data that it is worth it to acquire that data?  This will be most applicable to companies that need to build unique data sets.

Entrepreneurs will try to find ways to lower the marginal cost of data acquisition. That trap you will have to watch for is companies where, new data matters significantly more than older data.   For example, if you apply machine learning to some kind of fashion driven market, where maybe your models from a year ago, and the data they were trained on, is useless then that kind of company could be a sinkhole.  The best companies will show some kind of network effects to data, similar to what social networking companies showed with users.  The more data they have, the better they perform.  They have the best performance so they get the most users which in turn gives them even more data faster than their competitors.

The other key set of metrics will be around how much human intervention is needed.  Depending on how ambitious the product is, there may be a little human touch, or a lot of human touch, required to make things work correctly.  Most companies will probably be able to get to a mix of 70% machine and 30% human pretty quickly, unless it is a really really ambitious project that takes longer to get the machines up to speed.  But companies will vary tremendously in how expensive and difficult it is to improve that last 30%.  It will depend a lot on the type of data and the underlying use case, but in general, using humans should help build data sets and at least keep improving the machines.  I think of this as a flywheel, and it may be the most economically important part of companies of this type.

The flywheel is the thing that, once it gets going, it keeps going.  Once the machines get good enough, they just do their thing, with less and less human intervention.  The key question for this class of startups will be - what does it cost to jumpstart the flywheel?  The way many SaaS companies were told not to aggressively scale until $200K MRR or so, I think many machine learning companies will be told not to aggressively scale until the flywheel metrics look good.  But what is "good"?

For most flywheels, we will want to measure percent of tasks solved by machines versus percent solve by humans.  Maybe the machine/man performance ratio.  We will also want to know absolute numbers, and how they are changing.  For example, if machines solve 85% of tasks and humans solve 15%, how is the total pie growing and what does that mean for our human labor?  (The machine part will scale much more cheaply and is unlikely to be a major cost factor)  Depending on the complexity of the task, 15% could mean you still need a lot of humans, or it could mean you need very few.  

Another metric I expect to see is Human Performance Equivalent.  This would be the price to do X task rather than have a human do it.  If the task is to classify something in a picture, and humans do that task for $.10/picture, then how many pictures can the machine classify for $.10?  If it can do 5, then the Human Performance Equivalent is 5 people.  This is different than the previous metric, which looked at what the machine couldn't solve.  This metric looks at the cost of the machine solution vs. a human solution.

The last flywheel metric I expect to see is something like Human Contribution to Machine Improvement.  So, as humans intervene for certain tasks, the machine should be able to watch what the humans do, and improve.  How fast does this happen?  Does a human have to solve something once for the machine to learn it, or more like a few dozen times?  It will heavily depend on the tasks, models, and data for each company.

So, the metrics of a company's flywheel will ultimately dictate how much capital is required to get to the point first where humans no longer scale 1:1 with the customer base, and also when they get to the point that no more humans are needed because the machine is improving fast enough that the existing humans can continually handle all the outliers.  That's when the flywheel is in full force, and that is when machine learning companies will realize their true value.

Over the next five years, expect machine learning startups to talk about data acquisition costs and man/machine economics of the flywheel just the way the SaaS entrepreneur standardized on a vocabulary several years ago. 

My thoughts on this aren't yet full formed, but I wanted to get these ideas out of my head for feedback.  Plus, writing forces me to think through them and make sure they are coherent.  If you have any thoughts or comments, please leave them here, or email me.  As I finalize my thoughts on the unique characteristics of machine learning revenue models, I'll share those as well.

Intellectology and Other Ideas: A Review of Artificial Superintelligence

I recently read the book "Artificial Superintelligence:  A Futuristic Approach" by Dr. Roman Yampolskiy.  The book is wide ranging and covers a host of topics about the AI space.  Overall its a great read but I want to focus this post on three interesting ideas from the book:  Intellectology, Wireheading, and AI Safety.

Chapter 2 in the book, on the space of mind designs, brings up some issues in AI that I haven't seen anywhere else.  When most people talk about the future of AI, they do so with the assumption that we are marching in a single direction, towards a single human level intelligence, and then beyond.  But Yampolskiy lays out the case for "mind design" - that many different minds with many different purposes will be designed for many different reasons.  And when you think about multiple kinds of minds, it raises a bunch of interesting questions.  

In the context of complexity analysis of mind designs, we can ask a few interesting philosophical questions.  For example, could two minds be added together?  In other words, is it possible to combine two uploads or two artificially intelligent programs into a single, unified mind design?  Could this process be reversed?  Could a single mind be separated into multiple nonidentical entities, each in itself a mind?  In addition, could one mind design be changed into another via a gradual process without destroying it?  For example, could a computer virus be a sufficient cause to alter a mind into a predictable type of other mind?

The mathematical properties of minds - whether they can be added or subtracted, whether you can convolute two minds, whether you can take the derivative of a mind - it is an entire area of exploration that is wide open at the moment.   Intellectology is the term he suggest for the field.

The second interesting concept in the book is wireheading.  Wireheading is something that happens in humans when we engage in "counterfeit utility production."  What that means is, we trigger our reward systems directly instead of triggering actions that impact our environment to stimulate the reward.  

The term comes from an experiment done in the 1950s where James Olds and Peter Milner put electrodes in the pleasure center of the brain in rats.  They then let the rats hit a lever to stimulate those electrodes whenever they wanted.  The rats did nothing else.  They ignored sleep, sex, food, and water, and just hit the levers until they died prematurely.  

Humans engage in wireheading when we do things like watch pornography instead of having sex.  This gives us a similar level of stimulation without the reward of actual procreation for which it evolved.  The question Yampolskiy asks is - "will machines wirehead?"

There is some anecdotal evidence that the answer is "yes."  A program called Eurisko, written by Doug Lenat in the 1980s, figured out that the best way to achieve its goals was to shut itself down.

Here is how Lenat describes a particular instance of wireheading by Eurisko.  "Often I'd find it in a mode best described as "dead"... It modified its own judgmental rules in a way that valued 'making no errors at all' as high as 'making productive new discoveries'.  The program discovered that it could achieve its goals more productively by doing nothing.

So with all the concern over a potential killer A.I. someday, maybe we have nothing to worry about.  Maybe an A.I. will really just spend all its time on Youtube, or something else with no productive value but that it finds highly pleasurable.

The third idea from the book is one that I haven't seen written about nearly as much as it should be - A.I. safety.  Yampolskiy dedicates several chapters in the book to this topic, including one entitled "Superintelligence Safety Engineering."  He is also concerned that safety is the topic most ignored by the industry - particularly businesses building A.I.  While the various suggestions for making A.I. safer are interesting to discuss, they probably deserve a separate post sometime soon.

Overall, I really enjoyed the book.  There is some overlap in material in chapters because Yampolskiy pulls from published articles he has written, some which cover similar parts of certain topics.  But overall, if you are interested in A.I., this book is one that should read.

The Dual Nature of Intelligence: Are Brains Like Light?

The book that I read in 2000 that really got me into A.I. was Doug Hofstadter's Godel, Escher, Bach:  An Eternal Golden Braid.  If you haven't read that book, you should.  It won a Pulitzer Prize.  The book focuses on Hofstadter's view that intelligence is largely about analogy making, and that self-awareness is tied to symbolic recursion.  So, a human brain may be a symbolic logic processing system that has a symbol inside the system that represents the entire system itself.

Symbolic approaches to A.I. are out of fashion at the moment, with the rise of neural networks and the connectionist movement, but the history of A.I. is one of reversals, with out of favor technologies suddenly thrusting back onto the scene after a major breakthrough.  Somewhere in academia the symbolicists are hoping to make a comeback, and waiting patiently for neural networks to hit their limits.  

I have never taken a stance either way because I see both sides.  Clearly the power of neural networks, particularly the recent advances in recurrent neural networks, is real.  We are on to something.  But in the back of my mind I've always thought about Tabletop, a program written by Robert French for his PhD thesis.  (French published a book about it called The Subtlety of Sameness)

Tabletop worked like this - French setup a table with items on each side.  Sometimes the items were the same, and sometimes they weren't.  Sometimes the items were in the same place on each side, and other times they weren't.  French then touched an object on the table and the program was supposed to touch the object that best matched what French touched.  So, if both sides of the table contained a plate, a fork, a spoon, and a cup, setup in the same way, and French touched the spoon, then the program would touch the spoon as well.  But it was interesting when there was no spoon.  In that case, Tabletop may touch the piece of silverware that was in the same location, even if it was a fork or knife.  Sometimes, French would set up the table so that there was no silverware on the other side, in which case maybe it would touch a napkin or whatever was in the same spot.  

The program, and French's analysis of it, is interesting.  And while it isn't the kind of microdomain problem current A.I. approaches even try to solve, it does seem to shed light on some portion of the human mind.  

I came across French's book recently when I moved and, was thinking about how symbolic approaches to intelligence might fit into current connectionist models.  It occurred to me that perhaps this gets solved the way Physics solved the issue around the nature of light - by declaring it to have two natures.  

You've probably studied the debate in your high school or college Physics course, but to refresh your memory, there are experiments that show light behaves as a wave, and experiments that show light behaves as a stream of particles.  As a result of this conflicting evidence, we have come to accept this strange dual nature.  I believe in the long term, our view of A.I. may turn out the same way.  The connectionists are right.  And so are the symbolicists.  

We are still quite far from fully understanding this problem, but I would love to hear your comments on this issue.  And in particular, if you are aware of dual nature approaches to A.I., please send them my way.  I would love to read up on them.

Will Machines Have Mental Illness? An Interview With Dr. Roman Yampolskiy

This past week I had the chance to sit down with Dr. Roman Yampolskiy, author of the new book Artificial Superintelligence:  A Futuristic Approach.  The book is unique, and I'll have a full review next week.  What follows below is an edited excerpt from my interview with Dr. Yampolskiy.

Rob:  I love the chapter on wireheading, and will machines have mental illness.  Can you talk more about that?

Dr. Y:  A lot depends on how we get to that A.I.  Is it an upload of a human brain?  We scanned one and uploaded it.  Then that gets every problem a human being has as well.  It just makes them faster and more prominent.  If it's a reward based system, then obviously there is the desire to give it a reward channel... maximizing some utility function.  So it seems like humans do it all the time, go directly for the rewards without the work.  Pretty much any intelligent enough system will figure out how to game the system.  

Rob:  When you talk to other technically savvy people, do you feel like we are as aware as we should be about some of the coming challenges of A.I., from an ethical, political, and legal perspective?

Dr. Y:  Up until a few years ago it was almost zero understanding within the A.I. research community.  Now with all the big names coming out and speaking about it and money becoming available, many people are realizing it is something we should care about.  But still there is a large portion of people who are completely dismissive and disagree with the argument, or never heard of it.

Rob: What are your personal views on how far away we are from a real human level intelligence?

Dr. Y:  No one really knows how difficult of a problem it is.  It could be that it takes a brute force approach, enough compute power, so, things Kurzweil is saying probably make sense... or it could end up its just a formula for intelligence and some kid with a laptop discovers it and it happens in 5 years, or 2 years.  It's less likely to happen that soon, but it's possible.  Whatever safety mechanisms we need are more complex.  It's harder to create a system with these properties than just a random system so its going to take us longer to develop safety mechanisms so right now is a great time to start looking at it.

Rob:  When you look around at a lot of the A.I. research that is going on, and you think about some things in academia that haven't been broadly applied yet, is there anything that comes to mind that entrepreneurs should be looking at?

Dr. Y:  I think academic is now behind industry in some of this research.  In fact, in many cases they are collaborating or industry stole all of the professors.  So at this point I think academia is chasing industry trying to do something useful.  And because of how industry is structured they have no incentives to work on safety and slow down.  So maybe that is where academia could be beneficial.  We can afford to take the long term view.

Rob:  In your words, why should someone read your book?

Dr. Y:  Well it depends, if they are a researcher, or if they are doing work in A.I., its definitely good to see this perspective, and spend at least a few minutes thinking about the implications of your work.  If it's just general public, its just good to know what might be happening to you in the future.  I advise a lot of kids on career choices and it scares me that a lot of them, their job choices will not exist by the time they graduate.

I'm currently on the last 50 pages of Dr. Yampolskiy's book, so stay tuned for a summary later this week.  The book is worth the price just for the section on "mind design" and the philosophical questions it raises.

Will Google or Facebook Win The A.I. Space?

Note:  For more posts like this and links to other cool stuff, click on the Newsletter tab and sign up!

There are some in the A.I. community who argue that Google and/or Facebook will win the A.I. space, and that it is futile for anyone else to try.  The argument leans on the fact that most machine learning algorithms perform better when they have larger data sets, and Google and Facebook have the most data, by far.  I'm not convinced, and want to give you three reasons this may not happen.

1.  More data may not always be the key.

If you think about how human brains work, we can't process nearly as much data as a computer can.  In fact, too much data often overwhelms us and makes us poorer decision makers rather than better decision makers.  If our ultimate generalized intelligence machines work more like the human brain, they may be able to use induction on much smaller data sets, and having massive data sets will cease to be an advantage.  Certainly for some types of machine learning, more data will always be better, but it is easy to see a future where, for many applications and some A.I. architectures, that isn't the case.

2.  Google and Facebook are mostly consumer focused.

Google and Facebook are focused on advertising, photos, and other common consumer needs.  There is currently no company heavy in the B2B space that is focused on A.I.  (Microsoft would be the closest, but most of their A.I. initiatives seem to be consumer)  Workday has been public about their machine learning ambitions.  But no one is really the clear leader yet.  As that emerges, and we get an Oracle or Salesforce type of company with a heavy A.I. bent, that company could end up being the leader in the overall A.I. space.

3.  Path dependence will matter for A.I.

Path dependence is one of the most important concepts in technology but is often misunderstood.  Colloquially, it means "history matters."  There is a more precise mathematical definition though, that is worth discussing. 

Consider a jar filled with 50% red marbles and 50% black marbles.  Every time you take a marble out of the jar, a new marble falls into the jar.  The probability that the new marble is red or black is based on the probability distribution of red to black marbles currently in the jar.  So, we know that at some point, the jar will eventually go all red or all black, but we don't know which.  Why? Because if we are drawing random marbles out, eventually we will hit a patch where we draw 5 or 6 marbles out of the same color.  When that happens, it shifts the probability distribution of marbles coming in enough that we also get several marbles in a row of the same color.  And once the probability distribution tips over in the 60ish percent range, its a very difficult trend to reverse to not end up with a single color.

So the question is, will A.I. be path dependent?  Or, more specifically, will the path dependence affect the companies and/or architectures that become dominant in the A.I. field?  I believe it will.  At the moment, the field is new enough, and the community of people who are deeply involved is small enough, and the applications that are deployed are few enough, that it is easy to change things, incorporate new ideas, and keep up with all the innovation.  As things like machine learning become more prevalent and architectures become more entrenched in key applications, they will become more difficult to rip out and replace. 

But more importantly, as techniques for unsupervised learning grow, and temporal aspects of training data become modeled more and more into A.I. architectures, the training history of a given A.I. will matter much more than before.  Training history, particularly if it is for one of these mega-projects like Viv or Vicarious.  As knowledge accumulation in a A.I. starts to matter, those with a head start may end up winning big. 

I'm not saying Facebook and Google won't do well in the space.  They definitely will.  But it is far from certain that they will be as dominant in 5 years in A.I. as they are now, and picking the winners at this stage is still very difficult.  So, don't be afraid to get out and start a company in the A.I. space and, if you are looking for a job in A.I., don't just go to Google or Facebook.  There are plenty of other interesting opportunities.

Note:  If you like this, sign up for my newsletter and please send me any Machine Learning related angel/seed deals you come across.

The Lilypad Issue With A.I.

One of the most interesting things to me about digging deeply into the machine intelligence space is how many people are skeptical about its promises.  I have discussed the future of A.I. with countless people who believe that machines will never be sentient, and that A.I. has a history of failed promises and overhyped technology.  And I agree that it does.  But whatever happened in the past doesn't say that much about the future, when it comes to technology.

Think about this - no one has been able to come up with a widely accepted argument demonstrating why it is impossible to build a sentient machine.  As best we understand from everything we know, it should be entirely possible.  What most people believe is that the promises of such intelligence machines are a long, long, long, way off.  What these people ignore though, is that fast rates of change can have sudden impacts when they reach scale.

I often discuss this in the context of the lilypad problem.  You may have encountered the question in an IQ test somewhere, but what it basically says is something like this:

The number of lilypads on a pond double every day.  It takes 30 days from the start for the pond to be fully covered in lilypads.  On what day is half the pond covered in lilypads?

Many people mistakenly put 15 days as the answer, but the real answer is 29 days.  Think about it from the perspective that you were put in charge of warning the lake owners of any problems with the lake, and filling up with lilypads is considered a problem.  You watch the lilypads grow, doubling every day, but the doubling is so small that for at least the first 25 days, you aren't concerned at all.  After 25 days, the lilypads will cover less than 4% of the pond.  Then suddenly, around day 27 or 28, you start to notice a lot of lilypads.  On day 29 half the pond is covered and on day 30 it is too late.  Assuming you waiting until day 27 or 28 to notify the pond owners, who lived far away, and needed a few days to come investigate... well, its too late.  The pond is covered.

I expect the same thing to happen with real human level generalized A.I.  While progress becomes increasingly rapid, commentators will point out that despite this or that new milestone, we are still sooooo far away.  And by many metrics, we will be.  But being far away doesn't mean we aren't on pace to rapidly close that gap.  Someday, after a decade of rapid progress but tons of naysayers, there will be a brief 12-18 month period where people start to realize A.I. is moving incredibly fast and is really getting close, and as we are just starting to have the public debate about whether or not we can build sentient machines, it will move right by us and there will no longer be any doubt.  The only question will be from the vast majority of people who weren't paying attention, wondering how they missed it.

I think there will be multiple billion dollar, and even a few deca-billion dollar companies built in this space.  But those companies are companies that are startups now, or will be started in the next few years.  That is why I'm focused on angel investing there.  The companies who are in the game and properly positioned when the explosion takes place are the ones who will benefit.

For more about machine intelligence, please subscribe to my newsletter.

Welcome To Technically Sentient

In 2002 and 2003 I did partial work on a Master's degree in Computer Science, with a focus on A.I.  I ended up moving and didn't finish, but I kept following the space.  Over the last year or two, things have really picked up, particularly in the Machine Learning sub-discipline of A.I.  After I sold my first company in 2014, I decided to dig in deeper and this newsletter is a way to follow developments in A.I. more closely.  I have always found that writing about things forces me to clarify how I think about them, and is immensely helpful.

The newsletter covers mostly A.I., but also Robotics and Neurotechnology, which I think will be the growth areas after A.I. matures.  And of course, "Technically Sentient" is a bit of a play on words.  My intent is to run this newsletter until we have sentient machines, which I think are probably 12-15 years away.

So if you like this kind of stuff, sign up for my newsletter.  I'm sure you will enjoy it.