But we could have an average of 684 9/11’s a hour, the actual number of car crashes per hour in the US, driven purely by the piloting skills of the average Amarican driver given command of an aircraft, who wouldn’t want to live in that future?

“A computer can never be held accountable, therefore a computer must never make a management decision.”

Even more importantly when it comes to assessing properly, machine learning, now referred to as AI, has been continuealy shown to not just repeat the biases in its training data, but to significantly exaggerate them.

Given how significantly and explicitly race has been used to determine and guide so much property and neighborhood development in the training data, I do not look forward to seeing a system that is not only more racist than a post war city council choosing where to build new moterways but which is sold and treated as infallible by the humans operating and litigating it.

Given the deaths and disaster created by the Horizon Post Office Scandel, I also very much do not look forward to the widespread adoption of software which is inherently and provablly far less accurate, reliable, and auditable than the Horizon software. At least that could only ruin your life if you were a Postmaster and not just any member of the general public who isn’t rich enough to have your affairs handled by a human.

But hey, on the bright side, if Horizon set UK legal precedent than any person or property agent is fully and unequivocally legally liable for the output of any software they use, after the first few are found guilty for things the procedural text generator they used wrote people might decide its not worth the risk.

They are definitely to simple to represent the entirety of an concepts meaning on their own. Yep, I don’t believe it’s likely that such an incrediblely intricate thing as a nuron, much less the idea of conceptual meaning, can be replicated by a high school math problem. Maybe they could be a part, but your off by about a half a dozen order of magnitude at least from where we are now with love being a matrix with a few hundred numbers in it.

No part of a human or animal brain operates on subtracting tables of cleanly defined numbers from each other so I think it’s pretty safe to say that no matrix calculation is done on a handful of numbers as part of much less as our sole means of understanding concepts or objects.

I don’t know exactly how one could tell true understanding from minicry, far smarter and more well researched people than me have debated that for decades, i’m just pretty sure what we think an kindness is boils down to something a bit more complex than a high school math problem discribing a word cloud.

Generally the term Markov chain is used to discribe a model with a few dozen weights, while the large in large language model refers to having millions or billions of weights, but the fundamental principle of operation is exactly the same, they just differ in scale.

Word Embeddings are when you associate a mathematical vector to the word as a way of grouping similar words are weighted together, I don’t think that anyone would argue that the general public can even solve a mathematical matrix, much less that they can only comprehend a stool based on going down a row in a matrix to get the mathematical similarity between a stool, a chair, a bench, a floor, and a cat.

Subtracting vectors from each other can give you a lot of things, but not the actual meaning of the concept represented by a word.

To note the obvious, an large language model is by definition at its core a mathematical formula and a massive collection of values from zero to one which when combined give a weighted average of the percentage that word B follows word A crossed with another weighted average word cloud given as the input ‘context’.

A nuron in machine learning terms is a matrix (ie table) of numbers between zero and 1 by contrast a single human nuron is a biomechanical machine with literally hundreds of trillions of moving parts that darfs any machine humanity has ever built in terms of complexity. This is just a single one of the 86 billion nurons in an average human brain.

LLM’s and organic brains are completely different and in both design, complexity, and function, and to treat them as closely related much less synonymous betrays a complete lack of understanding of how one or both of them fundamentally functions.

We do not teach a kindergartner how to write by having them read for thousands of years until they recognize the exact mathematical odds that string of letters B comes after string A, and is followed by string C x percent of the time. Indeed humans don’t naturally compose sentences one word at a time starting from the beginning, instead staring with the key concepts they wish to express and then filling in the phrasing and grammar.

We also would not expect that increasing from hundreds of years of reading text to thousands would improve things, and the fact that this is the primary way we’ve seen progress in LLMs in the last half decade is yet another example of why animal learning and a word cloud are very different things.

For us a word actually correlates to a concept of what that word represents. They might make mistakes and missunderstand what concept a given word maps to in a given language, but we do generally expect it to correlate to something. To us a chair is a object made to sit down on, and not just the string of letters that comes after the word the in .0021798 percent of cases weighted against the .0092814 percent of cases related to the collection of strings that are being used as the ‘context’.

Do I believe there is something intrinsically impossible for a mathematical program to replicate about human thought, probably not. But this this not that, and is nowhere close to that on a fundamental level. It’s comparing apples to airplanes and saying that soon this apple will inevitably take anyone it touches to Paris because their both objects you can touch.

Like say, treating a program that shows you the next most likely word to follow the previous one on the internet like it is capable of understanding a sentence beyond this is the most likely string of words to follow the given input on the internet. Boy it sure is a good thing no one would ever do something so brainless as that in the current wave of hype.

It’s also definitely becuse autocompletes have made massive progress recently, and not just because we’ve fed simpler and simpler transformers more and more data to the point we’ve run out of new text on the internet to feed them. We definitely shouldn’t expect that the field as a whole should be valued what it was say back in 2018, when there were about the same number of practical uses and the foucus was on better programs instead of just throwing more training data at it and calling that progress that will continue to grow rapidly even though the amount of said data is very much finite.

Except when it comes to LLM, the fact that the technology fundamentally operates by probabilisticly stringing together the next most likely word to appear in the sentence based on the frequency said words appeared in the training data is a fundamental limitation of the technology.

So long as a model has no regard for the actual you know, meaning of the word, it definitionally cannot create a truly meaningful sentence. Instead, in order to get a coherent output the system must be fed training data that closely mirrors the context, this is why groups like OpenAi have been met with so much success by simplifying the algorithm, but progressively scrapping more and more of the internet into said systems.

I would argue that a similar inherent technological limitation also applies to image generation, and until a generative model can both model a four dimensional space and conceptually understand everything it has created in that space a generated image can only be as meaningful as the parts of the work the tens of thousands of people who do those things effortlessly it has regurgitated.

This is not required to create images that can pass as human made, but it is required to create ones that are truely meaningful on their own merits and not just the merits of the material it was created from, and nothing I have seen said by experts in the field indicates that we have found even a theoretical pathway to get there from here, much less that we are inevitably progressing on that path.

Mathematical models will almost certainly get closer to mimicking the desired parts of the data they were trained on with further instruction, but it is important to understand that is not a pathway to any actual conceptual understanding of the subject.

At first glance I thought it was reuseing the coal plants turbines, but looking though the article the only connection I can find is that it’s located several miles away and the only connection is that it plans to hire a hundred or so people from the coal plant it’s replacing and that Wyoming’s powder river basin is nearby and its associated highly automated low sulfur coal mines are in the vauge area.

All this to say, yes it has practically nothing to do with coal.

Then by definition the employees are perfectly compensated for the value of their labor, as that labor can only be worth exactly what the company pays for it, and there is absolutely no reason why they should get any more. That’s actually one of the reasons why the value = money theory tends to be pretty rare outside of the far right libertarianism.

Please explain to me how any of the child level explanation of the stock market is obfuscation, or again how you think the market cap, a purely theoretical number, could possibly be redistributed to employees outside of things the company already does to some extent, and finally why it applies in this case with a company who’s stock price is based purely on speculation about what it could do in the future and not anything it’s employees are currently doing.

Also from your comment about how share price literally is the only measure of value for a company I’m taking it you follow the theroy of value that value directly equals the amount of money paid for it, which seems inherently contradictory to this entire conversation.

Technically, they don’t even make the actual graphics cards, they just design them and then outsource manufacturing to TSMC.

But don’t you know that doesn’t matter, because by 2028 every singe company in the world is going to need a data center filled with tens of thousands of AI accelerators turning their own scrape of the internet into a chatbot, and so one of the companies that makes thouse accelerators is definitely going to have as much business as companies that make half of everyone’s phones or computer software./s

Firstly it shows the value of individual shares multiplied by the number of shares, not the company as a whole. Secondly, in this case Nvidia’s share price is based on what the company may be able to expand to do in the future, not what it currently does. Thirdly, where would this repersentive percentage come from? If it’s, issueing new stock to employees, A Nvida already does that a lot, B, creating new stock is not practically reliant on overall market cap so why is it relevant, and C, would employees also be punished for destroying the valuation if it turns out that every company doesn’t actually need a data center full of several thousand AI accelerators scraping the internet to make unique chat bots and Nvida’s market cap falls back down to what it would be based on how much money the company actually makes?

Again, Nvida primarily makes chip designs for outsourced fabrication, not market cap, that three trillion isn’t like revenue for Nvidia. In your painting example, market cap would be like if two unrelated billionaires bet 10 billion on whether or not that painter would be successful in selling a hundred different 1m paintings in the next six months, the painter might have an easier time say getting a loan for new supplies from a bank if they can point to the billionaire betting so much on them, but you know it’s not like the painter was actually paid that 10 billion that makes up the bet, right? So it’s kind of weird to say that the painter’s work as a whole is definitely worth that 10 billion bet.

I’m saying that while a companies market capitalization is a real number that can tell you things about a company, it is not like anyone involved has a three trillion actual dollars. The company doesn’t see any of that money directly unless they directly issue more stock which would devalue the current stock, though there are some other ways for a company to use it to their advantage. Investors might be able to get a small percentage of that by selling, but only because someone else bought in with an equal amount of money, and a large sell will drive down the price.

More to the point, the evaluations people are doing with Nvidia don’t have much to do with what the company actually produces and puts out into the world today, but the assumption that it can turn its current leadership position in AI accelerator chip designs into growing massively in size in the future when every company needs a large data center or two to train their own individual LLM’s.

A individual stocks price is driven primarily by what people think that individual stock certificate can be sold for in the future, and effected by things like how many people are trying to sell, adding all of those certificates up at current market price doesn’t actually give anyone involved much information, nor does it reflect the actual quality, quantity, material, or labor taken to make things, in this case branded computer chip blueprints, that a company puts out into the world.

Now there are a lot of competing theories of ways to try and measure labor’s value, but my work being only as valuable as the speculative amount my organization as a whole might be theoretically sold for as a whole in the future if no one tries to undercut anyone else isn’t one of the more popular ones.

I don’t think many people would claim overall valuation has much of anything to do with the value labor brings to an organization.

In this case I think all it indicates is just how much the company’s stock price is driven by speculation about possible demand for generative AI, and even then I’m not sure that current price per share times number of shares divided by number of employees is a clear indication of that.

If it makes you feel any better, modern climate and economic studies have shown that even a full scale nuclear war involving every nuclear power at the height of the Cold War and when nuclear stockpiles were far larger than today we still wouldn’t have come very close to actually killing off all the humans on earth, with the vast majority of the casualties being owed to famine in regions that were/are heavily dependent on western fertilizer. Indeed entire nations in the southern hemisphere tend to get through such senecios without much of an direct effect from world war three.

Mostly this change from earlier predictions came from being able rule out the theory of a nuclear winter as climate modeling became more accurate and we could be sure that the secondary fires from such a war could not carry ash into the upper atmosphere in significant quantities, which was practically shown when a climate change fueled wildfire in Australia got so large that it should have been able to carry the ash into the upper atmosphere under nuclear winter theory but none was observed, validating modern climate models.

Also, dispite what some less scrupulous journalists trying to drum up clicks have posted on the Ukraine War, the Russian government itself hasn’t really made any major signaling moves with regards to bringing nukes into the conflict, and indeed has maintained and repeatedly reiterated Putin’s 2010s no first use policy when asked.

Don’t get me wrong, this is not the result of some greater Russian morals or whatever, but just a consequence of the inherent risk that such posturing could lead to nuclear escalation and breaking the nuclear taboo or even just other nations actually believing they plan to, and such scenarios end very badly for Russia in general and Putin in particular.

Opinion pieces on the Internet and political saber rattling by low level politicians does not a nuclear policy make.

States actually have quite a few different ways of signaling they are serious about potentially ending the world as we know it, and Russia is currently using none of them.

As an example, the Russian state’s own published nuclear policy has remained unchanged for over a decade and still explicitly prohibits nuclear first use in cases like this. Currently high level Russian politicians including Putin continue to reference said defense policy in response to questions about the use of nuclear weapons in Ukraine. If they were seriously considering using said nuclear weapons in Ukraine, they would be unambiguously signaling through changing these documents and other such methods that other governments actually take seriously.

More to the point, breaking the nuclear taboo would be massively harmful to both Russia and Putins own interests. It would at best result in a NATO backed no fly zone over Ukraine while China and Iran completely abandon them, and quite possibly result in a direct conventional or nuclear war with Nato. I simply don’t buy that they would do that with no warning or previous signaling simply because an artillery rocket was manufactured in a different country.

I mean, the government has mandated that all cars built since the 90s have to have a lot of computers and sensors for engine monitoring and emissions logging so that ship has long since sailed. Automatic braking is also credited with eliminating something like 1 in 5 fatalities in car accidents, so as long as we have any motorized vehicles around at all I don’t really have a problem with the government requiring manufacturers to spend the extra 20 dollars or so per vehicle it costs them to add a few ultrasonic sensors and a microcontroller it takes to slow the vehicle to the point where a driving into a pedestrian might just be survivable.

While I think in this case they won’t have an effect because no Amarican company is even trying to compete in the space, I feel like claiming “history says tarrifs rarely work” is pretty misleading. The high tarrifs caused by the US generating nearly all federal income by tarrifs in the 17 and 18 hundreds are after all widely credited with being the reason the northern US went from being a minor agricultural nation dependent entirely on european industrial goods to becoming one of the largest industrialized nations so quickly.

Indeed that was why the WTO blocking third world nations from putting tarrifs on western goods was so heavily criticized by the left a few decades ago, before China proved you could do it without said tarrifs so long as your competitors were greedy enough to outsource their industry to you.

While the paper demonstrated strong diminishing returns in adding more data to modern neural networks in terms of image classifers, the video host is explaining how the same may effect apply to any nureal network based system with modern transformers.

While there are technically methods of generative AI that don’t use a neural network, they haven’t made much progress in recent decades and arn’t what most people mean when they hear or say generative AI, and as such I would say the title is accurate enough for a video meant for a general audience, though “Is there a fundamental limit to modern neural networks” might be more technically correct.