AI finds thousands of zero-day exploits... including in FreeBSD.

kpedersen said:
Interesting but it should be noted that the costs are completely arbitrary and are not dictated by any kind of technical limitation. Next week they could be completely different. Higher or lower.

In a gold rush, the costs are dictated by the guy selling the shovels.
Click to expand...
In the short term, the firms selling AI services can set the prices, but the economic costs of compute are something they do not control. (In fact they'd prefer those costs to come down, even as their demand for hardware and energy is pushing costs up.) The quoted figures were based on API pricing, which as you say is somewhat arbitrary. But like I said, I think it's fair to assume the true costs are currently higher - AI firms are prone to underpricing tokens as they chase market share and customer adoption/dependency. At some point that underpricing is surely going to end as investors seek a path to profitability. While it's fair to treat Anthropic's quoted figures with a pinch of salt, I doubt they're totally disconnected from reality. They probably serve as a decent lower bound for the true costs. Even looking at things cynically, it's not exactly in Anthropic's interest to admit their model is a very expensive way of hunting bugs.
 
BigSneakyDuck said:
In the short term, the firms selling AI services can set the prices, but the economic costs of compute are something they do not control. (In fact they'd prefer those costs to come down, even as their demand for hardware and energy is pushing costs up.) The quoted figures were based on API pricing, which as you say is somewhat arbitrary. But like I said, I think it's fair to assume the true costs are currently higher - AI firms are prone to underpricing tokens as they chase market share and customer adoption/dependency. At some point that underpricing is surely going to end as investors seek a path to profitability. While it's fair to treat Anthropic's quoted figures with a pinch of salt, I doubt they're totally disconnected from reality. They probably serve as a decent lower bound for the true costs. Even looking at things cynically, it's not exactly in Anthropic's interest to admit their model is a very expensive way of hunting bugs.
Click to expand...
Exactly. Right now the AI companies are under-pricing their products to gain market share by burning venture money. Eventually some small number winners will emerge and prices will begin to rise to match costs.

I have no idea how much they will rise. I suspect quite a bit because all of us are used to a world where Moore's Law applies. That is no longer the case. There's probably room for optimization that might lower costs, but the need to scale up the models ever higher runs in the opposite direction.
 
aphor said:
Is this truly a need, or is it an illusion projected by parties with a conflict of interest?
Click to expand...
Supposedly, scaling them up led to a huge increase in their usefulness. That improvement seems to have plateaued at GPT-5, and the gains now are incremental.

But consider I got all of the above from a Youtube video by an infamous AI skeptic. I'm certainly not qualified to evaluate the validity of those claims. He seems credible to me, for whatever that's worth.
 
rbranco said:
There's lots of hype but there's also doomerism and both sides are wrong.

The most obvious use-case for AI is cheap entertainment like memes and NSFW content that even OpenAI is flirting with.

Those waiting for an AI bubble pop will be disappointed... We don't need AGI for LLM's to be useful. The tech is here to stay.
Click to expand...

Jose said:
Supposedly, scaling them up led to a huge increase in their usefulness. That improvement seems to have plateaued at GPT-5, and the gains now are incremental.

But consider I got all of the above from a Youtube video by an infamous AI skeptic. I'm certainly not qualified to evaluate the validity of those claims. He seems credible to me, for whatever that's worth.
Click to expand...

I'm watching Yann Lecun and his JEPA stuff.

Also, there's TONS of utility in other machine learning algorithms. IMO, the real value creator here is smashing the cultural insistence that everyone needs a lot of 100% certain data to make decisions. In my career I have personally counted hundreds of millions of dollars wasted on technology that makes people who can't make consistently good inferences feel more confident in their decisions because of the certainty and volume of supporting data. Getting people to accept that nothing is actually certain and that making good gambles over time is the success path will lead to efficiencies and probably better outcomes besides as people become aware of confidence as a tunable parameter.
 
As you've noted, that's not how people used the technology in the past with more perfect data and I don't foresee them doing any better with less perfect data just because somebody slapped a LLM disclaimer on it.
 
msplsh said:
As you've noted, that's not how people used the technology in the past with more perfect data and I don't foresee them doing any better with less perfect data just because somebody slapped a LLM disclaimer on it.
Click to expand...
There will be losers among the many no doubt.
 
BigSneakyDuck said:
Anthropic's Mythos Preview writeup from 7 April was pretty upfront in several places about the cost of finding their exploits, at least in terms of API pricing (the true cost may be higher, of course - but also bear in mind the long-term trend of reducing compute costs so these figures are likely to become more affordable eventually)
Click to expand...

The long term trend of reducing 'compute' costs was primarily fueled by Moore's law which ceased to be valid.
Company such as Anthropic runs (at minimum) two types of server farms, one for runtime and other for training. And they don't run them under market terms, paying the real industrial power and water bill. Not only is the cost of operation covered by outside capital, that cost is artificially kept on the floor.

I don't think they're even betting that 'compute costs' will go down because that's not the way it goes. Take a look at number of transistors in last 5 years in CPUs and GPUs, we are not growing anymore.

There is no dramatic conclusion to this, like the dotcom crash. Eventually the amount of venture money going towards AI is going to calm down, and the amount of sweetheart deals they're getting, not to say corruption, from local officials giving them land power and water is going to go down, at that point the prices will start converging to what I deem as 'realistic market cost'.

It is the same market cost that you and I would have to pay if we decided to get into AI fairly. Sit in line for GPUs, pay huge rebuffs to local community where we built the datacenter, prop up their infrastructure so our water/electricity demand doesn't affect them and last but not least ensure we have fair usage over all training data used.

Right now it is not possible to say how much does a token really cost.
 
Zare said:
The long term trend of reducing 'compute' costs was primarily fueled by Moore's law which ceased to be valid.
Click to expand...
For training I don't know, but for running the LLMs, there are various projects for dedicated processors, that are more efficient than GPU. So costs can go down by better hardware design rather than Moore Law.
 
OpenFreeNet said:
For training I don't know, but for running the LLMs, there are various projects for dedicated processors, that are more efficient than GPU. So costs can go down by better hardware design rather than Moore Law.
Click to expand...
The irony I see is that the current crop of LLM transformer networks grew out of Goffrey Hinton's 2007 deep learning successes, which he attributed to the combination of Moore's Law and the realization that it's not necessary to do exhaustive gradient descent at every step.
View: https://youtu.be/AyzOUbkUf3M?t=472


Then he went to Google to found DeepMind, where they turned it into a massive parallel compute thing so that they could promote tech they dominate (moar racks of moar 'puter).
View: https://youtu.be/d95J8yzvjbQ?t=4197
 
aphor said:
IMO, the real value creator here is smashing the cultural insistence that everyone needs a lot of 100% certain data to make decisions. In my career I have personally counted hundreds of millions of dollars wasted on technology that makes people who can't make consistently good inferences feel more confident in their decisions because of the certainty and volume of supporting data.
Click to expand...
And I've watched hundreds of millions of dollars wasted by people who abused data and inference to agree with their confirmation bias. If there's one thing AI does well is confirm your biases.
 
OpenFreeNet said:
For training I don't know, but for running the LLMs, there are various projects for dedicated processors, that are more efficient than GPU. So costs can go down by better hardware design rather than Moore Law.
Click to expand...
And aside from design, hardware costs can also fall due to economies of scale and improvements in manufacturing techniques (what economists call "learning by doing"). In the past, a significant driver of the downwards trend in cost of installing solar power generating capacity was the ever-improving efficiency of solar panels. Today panel efficiency is almost at a plateau as designs approach the theoretical limit, but costs are still falling rapidly as production expands. Cost of compute may follow a similar route - the global demand for ever more computational power is certainly there - even if costs don't drop as quickly as during the peak of Moore's law.

For many practical applications - perhaps not so much AI where the number-crunching has been heavily optimized over the years, unless someone comes up with a new Big Idea that rethinks how to do it more fundamentally - there's also a lot of room for more efficiency on the software side of things. It's easy for applications to get flabby if you treat hardware costs as minimal. If it becomes important to eke more performance out of the hardware, perhaps at greater development cost, then the opportunity is there.
 
Jose said:
And I've watched hundreds of millions of dollars wasted by people who abused data and inference to agree with their confirmation bias. If there's one thing AI does well is confirm your biases.
Click to expand...
But how does that make you FEEL?
 
MG said:
NFS on top of iSCSI? Never used it but nfs doesn't really care about the physical medium of local files. It suits for quick access to a network directory with only a few commands and it needs no reboot to activate it.
Click to expand...
iSCSI and NFS both initially gained significant traction in the data center and corporate IT realms, where users applied it as a backing file system for a long time. They mainly connected them to storage area networks (SANs), which are completely isolated , from the corporate extra and intra nets let alone the wider Internet, if your organization's IT department even knew the difference between a DHCP assigned IP address and a statically assigned IP address.
 
You must log in or register to reply here.
Back
Top