Evil hacking using AI vs. AI self-defense

If AI tools are so great for interrogating and exploiting Internet services with related apps, then why aren't there AI utilities to defend against them. Btw, what happened to the co called "self healing systems", the hypetch that was trending many years ago? And, I'm waiting for an update or upgrade of the TCP/IP protocol. I have not updated or upgraded my TCP/IP since 1995! I don't even know how to find out what version of it I'm running on right now :D
 
All the hype surrounding AI will, one way or another, eventually settle. What will remain on the surface—I don’t know. I also don’t know how to check the version of a network stack—but then again, why would I need to?
I suspect that AI is being "fed" to the masses in measured doses. Those who control AI know exactly what they’re doing, and they are currently using it to great effect for manipulation. In the midst of this entire race, the most important thing is not to lose one’s own intelligence. I believe an "antidote" to AI exists, but it’s being kept from us. Their goal is to establish a new societal framework by 2030—one with a rigid, predetermined trajectory of development. I, too, will likely be caught up in this trend to some extent. FreeBSD will be "pushed" to adapt, and AI will be integrated into both kernel space and user space. In short, this is a very long conversation...
 
What do you expect?
They are not even capable to verify their own answer.
One of my favorite examples:
Last year I asked ChatGPT, 'On which day will be this year's thanksgiving?'
After a long text about the history of thanksgiving I did not asked for, and telling me a couple of times that it is always on a thursday, this shit actually really closed with the only sentence I was interested in:
'This year's thanksgiving will be on friday the ...' - blablabla wrong.

Straighforward alone is not always the best choice.
One needs to reconsider to do no mistakes.
In control engineering it's called a control loop: The output is feeded back to the input to compare the result with the instruction.
Either somebody stands by a system, watch it, and control and correct it if needed, or it must consist of an reliably working, automatic control loop plus watchdogs and emergency stop measures.
Otherwise any system gets out of control sooner or later - doesn't matter if it's a steam engine or a space ship.

Above all it's also kind of a marketing stunt: 'Hey, look how powerful our thing is!'

Apart from that, it's "good old" US american tradition to give right of way for anything money can made with, until there are enough complaints, laws broken, and lawsuits. Which, when I got it right on yesterday's HN California started right now, which could be put those unleashed AI freaks into their urgent needed limits.
 
What if AI defenses detected AI bad actors and the AI decided to help the AI?
I was thinking about that too. What if AI driven military weapon systems started to hallucinate and turn against their own fighting force. Wouldn't that be great? That's what happens often in the real life during military conflicts when the war fighters start to hallucinate. AI algorithms follow the same logic when implemented by hallucinating scientists and software developers 😁
 
Apart from that, it's "good old" US american tradition to give right of way for anything money can made with, until there are enough complaints, laws broken, and lawsuits. Which, when I got it right on yesterday's HN California started right now, which could be put those unleashed AI freaks into their urgent needed limits.

Ya - "The Silicon Valley" logic is: "Go fast and break everything !"

And well... that is what is happening !

To give FreeBSD (A LOT) of credit - I do think FreeBSD is better able to handle the "AI shock and awe" that is going on right now. The O/S's that are truly not built WELL from the very bottom ground to the top are going to have a terrible time with the AI shock and awe that is going on. FreeBSD has many many years of having a solid BASE foundation that is completely understood and can be repaired and deployed to all of us. The duct taped together O/S's are going to have a lot of fun playing constant "whack-a-mole" with AI.

The "Do ONE thing well !" philosophy of UNIX is going to serve our FreeBSD community very well.
 
Complete source code review "by humans" is going to be very important in FreeBSD going forward.
At this point in time most humans are not able to review code generated by others or AI. Only humans who posses Real Intelligence from another planet might be able to review computer code created by other humans or AI 😁
 
If AI tools are so great for interrogating and exploiting Internet services with related apps, then why aren't there AI utilities to defend against them.

Those capabilities are completely unrelated to each other. Offensive security is different from defensive security and programming. AI even writes better exploits because most exploits contain sloppy code. There's human slop and AI slop (which may be due to the training data).

The truth has been always somewhere in the middle. Both AI psychotics and nAIsayers are wrong. The synergy of human natural intelligence and artificial intelligence will always outperform clankers' and humans who don't leverage AI.
 
I do think FreeBSD is better able to handle the "AI shock and awe" that is going on right now.
Me either.
Complete source code review "by humans" is going to be very important in FreeBSD going forward.
Not only here.
I may be mistaken, but I cannot help the feeling that in a soon future it will be said, 'It was a bad idea to let all this AI generated code into our sources. And those who didn't, or were more careful about this, now are the bride grinning winners, telling us, "We told you so." It's nice to use it for some tests or some prototyping etc., but committing AI generated code to the actual source tree in the end caused more trouble and work than it saved. We cannot bring the project further. Not at the moment, not in the next three years. Because of AI a lot many programmers have been fired, and now all our work capacity is bound only to find and remove that stuff from our source. There is so much written in so short time, it's a giant heap to clean up. Almost none of it was clearly marked as "made by AI", but accepted equally, so finding and discriminating it alone is a herculean task of its own.' 'Maybe AI can help on this', says the class clown and is shot for this on the spot. 'It's pointless. It cannot be done. We need to go back and fork freshly new all over again from a ten years old version.'

Maybe it's I tell the bots the wrong things about what exactly I want/need. But also programming the AI and learning how to has to be substracted from the presumptive saved time - additionally to learning to program and the according language, 'cause otherwise you cannot review and validate it.
What I get then is either something 1:1 copied from somewhere I could have got it by a traditional search by myself quicker. Or it's some long, weird, complicated, hard to read, even more hard to comprehend piece of ... something, that's not even remotely to the idea like I had written it, even worse nor does it look like any code I've seen written by far better programmers like myself (unless we are talking obfuscation.)
Plus seeing also the garbage it produces on questions from daily life, it's simply impossible for me to imagine this will come all in all to something useful but end only in some bubble burst.
I mean, when it tells me, thanksgiving is always on a thursday, and in the very same answer it says this year's thanksgiving will be on friday, what error produces it on code? For sure not syntax errors. There I'm sure it's all aces. But every programmers knows, just because a piece of software compiles errorless, runs, and does what it shall do doesn't mean it's free of errors.

And additionally there is also the not so pretty clearly solved point about the copyright of all the code where AI got its "knowledge" from.
Well, we will see.
As long as there are old sources stored savely in save places, and some projects refuse AI code, there also will be experienced programmers in the future, and it cannot get all lost.
 
Those capabilities are completely unrelated to each other. Offensive security is different from defensive security and programming. AI even writes better exploits because most exploits contain sloppy code. There's human slop and AI slop (which may be due to the training data).

The truth has been always somewhere in the middle. Both AI psychotics and nAIsayers are wrong. The synergy of human natural intelligence and artificial intelligence will always outperform clankers' and humans who don't leverage AI.
A significant security problem related to AI is an obscure process provided by a commercial entity taking lead authority.
Any AI system that is able to intrude into a foreign system is also able to generate a security audit that covers that method. The related company can choose not to and profit on the intrusion.
 
A significant security problem related to AI is an obscure process provided by a commercial entity taking lead authority.
Any AI system that is able to intrude into a foreign system is also able to generate a security audit that covers that method. The related company can choose not to and profit on the intrusion.
That's why open-source AI is so important. People with Vegan anti-AI views don't contribute anything to make it happen.
 
That's why open-source AI is so important. People with Vegan anti-AI views don't contribute anything to make it happen.
No need to care. In the 10 years of "commercial AI". it didn't produce 1 thing that has actual value. But ok, if you see the lack of interest as anti. I'm anti. No idea what being vegan has to do with it. It's on the same level, though: nobody cares what you eat.
 
No need to care. In the 10 years of "commercial AI". it didn't produce 1 thing that has actual value. But ok, if you see the lack of interest as anti. I'm anti. No idea what being vegan has to do with it. It's on the same level, though: nobody cares what you eat.
10 years?

Value is a subjective action we as individuals do everytime. Research about the Subjective theory of value. I won't point to the Wikipedia page because it frankly sucks.

This simple page explains it in nice layman's terms:
 
I have not updated or upgraded my TCP/IP since 1995! I don't even know how to find out what version of it I'm running on right now :D
Disable IPv6 on WAN and find out :p (iirc my modem says MDD IPv6 but still hands out IPv4)
Last year I asked ChatGPT, 'On which day will be this year's thanksgiving?'
After a long text about the history of thanksgiving I did not asked for, and telling me a couple of times that it is always on a thursday, this shit actually really closed with the only sentence I was interested in:
'This year's thanksgiving will be on friday the ...' - blablabla wrong.
Meanwhile the energy involved could probably fell a tree and I can walk over to a calendar :p
 
For why, see Gartner hype cycle. Maybe one day AI will be a great tool. Today it's just a lot of people going "look, I can do things with AI that I couldn't without!"

Ironically, both vibe-coders and AI-haters set the highest expectations. Vibe-coders blindly trust it and AI-haters dismiss it because it's non-deterministic. Both are tiny minorities at their peak.

Machines have strengths that humans lack and viceversa. We also commit different kinds of mistakes... So a synergy between natural & artificial intelligence makes sense.

I've never seen AI commit this kind of copy/paste errors:

The above could have been prevented if QEMU implemented rigorous LLM reviews. We reached the stage where no one can deny that LLMs are very good at finding bugs. They proved their worth already. Now we need open-source AI and cheaper hardware prices.
 
If FreeBSD were to have a handwritten FreeBSD university curriculum I would sign up yesterday

Universities used to teach an operating systems course with UNIX -- they actually used Andrew Tanenbuam's MINIX operating system -- and MINIX ironically later became the basis for the "Linux Operating system". (Link: Wikipedia): MINIX. You can get the complete source code to MINIX, and of course Linux from various Internet sources. Tanenbuam also has course books you can reference (See Amazon).

That said - the source code to FreeBSD is "freely available" and one can: view, edit, change and otherwise compile and run the finished FreeBSD kernel on your hardware. I actually used to edit and compile my own custom FreeBSD kernels back in the FreeBSD 12.x time frame for what I was working on at the time. You will find the source code to FreeBSD on most systems under: /usr/src

And you can install the source using pkg:

[15.1-RELEASE-p7]
shell $ pkg search FreeBSD-src
FreeBSD-src-15.1p7 System userland source code
FreeBSD-src-sys-15.1p7 System kernel source code
shell $

If you want a guide to the *BSD kernel -- (or) -- you just want to write and compile your own customer FreeBSD kernel driver - you can reference these books:
  • The design and implementation of the 4.4 BSD operating system - Marshall Kirk McKusick, Keith Bostic (and others)
  • FreeBSD Device Drivers - Joseph Kong
But reading the FreeBSD source code, changing it and seeing it compile and run --- is pretty cool ! :-). Be sure to do that on a VM, spare hardware, or similar -- so that you can crash your FreeBSD system and not care about it.
 
why aren't there AI utilities to defend against them.
Defense is more difficult than attack.
Using Ai for IDS/IPS is as of right now pretty expensive - a lot of tokens/wattage.

It could be mitigated by very secure architecture - basically leaving the least minimum in the open. And even then run everything through uni-directional gateways inspecting every packet and/or validating the inputs. Containment.
And the machines facing the Ai kiddies should be able to restore from fresh bare metal install.

But let me tell you nobody wants to sell Data Diodes.
 
Defense is more difficult than attack.

This is true.

AI doesn't care HOW it breaks something. If the AI breaks something "by accident", even doing something that the AI did not specifically intend to do, then AI is still successful. If AI break ends up breaking it's target into "10,000 different pieces" -- AI does not care, because AI doesn't have to put all of those pieces back together again, someone else (likely a "human") will end up doing that. In short AI has nothing to lose.

That said those on defense have (a lot) to lose when the AI breaks something.

We reached the stage where no one can deny that LLMs are very good at finding bugs.

It's a valid point that AI's are very good at breaking things. This (is not specifically "bad") because although serious bugs (and sometimes VERY serious bugs) are quickly found by an AI -- the (human Attacker?) that is watching the AI successfully break something -- will have to make a choice:
  • The AI has found a serious bug in product or library X
  • Does the Attacker "attack" product or library X NOW to get what Attacker wants ($$$, reputation, status, bug bounty, etc)
  • (OR) does the Attacker WAIT until a date/time when the attacker can maximize the benefit of attacking product or library X using the found bug.
  • For Example: Attacker waits for a "holiday weekend" to attack product when they are more likely to exploit the vulnerability and the bug won't be fixed or patched quickly.
In the past, Attackers (might?) be able "bide their time" and exploit each found weakness/bug when they liked, perhaps waiting day(s), week(s), year(s) etc and perhaps "formulating a plan" to exploit each found bug to it's fullest.

But with AI - (any) AI can discover (any) weakness in (any) product or library at (any) time. So now each "attack surface" has a much smaller shelf life now - either the "Attacker attacks RIGHT NOW (or as soon as possible)" and hopes for the best, or the Attacker can wait and perhaps someone else (human or AI) WILL ALSO discover the same bug and then patches it -- and then the Attacker will end up missing out on a given "attack surface" altogether.

So the "good news" is: AI puts a "real time timer" on each found bug. Either exploit the bug NOW or run the risk of never getting to exploit it in the future.

Using AI to "code" things is a terrible idea :cool: -- mainly because a "Human" ultimately has to review each code change before those changes ultimately go to production. This is basically what a recent Slashdot article was saying - basically the bulk of project coding has now moved from actual programming to "code review". The same thing as saying "humans" will review all the code that AI writes. How WELL that goes - nobody knows. (Link: Slashdot): AI Coding Agents Generate More Code, But Not More Software
 
Back
Top