Wöchentliche KI-Updates: Vulkan, Mojo, Kimi K3 und neue Agent-Frameworks

artificial intelligence updates every Friday at 2:00 p.m. Eastern time. Today, Friday, July 17th, uh you see a lot of updates. The epigraph today is uh Vulcan and Mojo are challenging Nvidia CUDA. So, we'll be talking about this new technologies. Uh okay. Uh these are the leaderboards from yesterday. uh July 16th and you see that uh in these leaderboards Claude is leading as usual. I want you to see the Kimmy uh 3. Uh so this this is actually new model. It's open weights. It was just released day ago. It is very big 2.8 eight uh trillion parameters and uh yeah it's on the level uh in fact in some leaderboards it's on the first place so it's a claude level I would even say claude fable level model and it's open it's quite quite amazing uh next um I recommend you this website which is called artificialanalysis.ai AI it has many charts and this particular chart is cost per intelligence index. So you see that uh clo fable which is the best right but it's also very expensive uh you can get same task same intelligence performed at a much cheaper price for example Kim K3 you see it's less than a dollar whereas for claude fable it's almost $3 right for GPT56 it's only4 45 from Muse Spark from Meta, it's only 26 cents. And if you go here in Deepseek V4 Flash and Deepseek V4 Pro, which are very good models, but you see we're talking about 2 cents or 4 cents, compare it with almost $3 for performing the same task for the same intelligence. It's it's a very good just go to this website and look at their charts. very eyeopening. Okay. Metam use spark released a new uh like it was version one now it's 1.1 uh multimodel reasoning. So this is uh from meta highly competitive outperforms Gemini 3.1 Pro and uh bits expensive like OPUS 4.8. Yeah it it it is on the top group of models in the benchmarks. uh 1 million uh tokens uh context length uh supports parallel agents across applications excel and computer use tasks and so on. So this is a very very good model but it is cheaper much cheaper than cloud code as you have seen in the previous diagram. Hermas cloud allows to deploy agents to a VPS in 60 seconds. So Hermas actually the company behind it news research they're now looking for funding 75 million at 1.5 billion valuation. So this uh company uh was formed several years ago uh out of group about 20 people and uh they created a a lot of stuff but the big success coming from the Hermas agent and now Hermas cloud and uh yeah Hermas AI update segmented tool batch dispatch allowing parallel agents workflows to run much faster instead of executing tools one by on they work in parallel and when they're blocking situation they just uh stop and wait until they will get unblocked but anyway works uh much faster than before okay uh so news research is an open-source AI lab uh this is the CEO of news research founded in 22 as anonymous online research community formally incorporated in 23 which is 3 years ago about 20 person team led by uh Jeffrey and uh yes so they have uh products like uh distro distributed training over the internet so you have multiple computers over the internet those computers have GPUs and these GPUs can be utilized together to do something useful like for example uh pre-training of the model and the Hermas agent was released It's just this year in February and this is their huge success. It's it's it's a very good system. Okay. Uh company raised 70 million total so far and now they're looking for another 75 uh million. Okay. Uh that's it. Uh next uh Prism ML so can run 27 billion parameter model on iPhone and they effectively compress it to about four to six uh GB size and the way they do it they do compression uh selectively and uh uh okay it was running iPhone 11 tokens per second which is pretty good uh compressed mode retains 90 to 97% of original quality and supports 262,000 token context length which is good supports speculative decoding via the spark and uh yeah so so this is very very good technology right um so prisml this is the website prisml.com um yeah okay next thinking machine Inkling. So Inkling is the name of the model and you see here on one of the benchmarks they're on pretty respectable position. So here we have Claude, Fable, Oppus and uh yeah uh so this company uh thinking machine uh labs it's Mira Morati. She is a former CTO of OpenAI and she created her own company got billions of dollars in funding and here they created this model. So this is almost trillion parameters. It is mixture of experts. It is multimodel and uh you can ask it actually to tune itself and it will do it. So if you have your own data, you can give it your own data and tell it tune yourself on this data. Uh so it's available on Tinker, datab bricks, Unity, AI gateway and on some other platforms and uh it is competitive and it is definitely uh cost efficient. Well, when you're running a trillion dollar parameter model, you need some serious hardware. So, it cannot be free, but it is of course much cheaper than running Oppus or something like that. Okay. The big event this week is Moonshot AI's new model, Kim K3. Well, they used to have uh Kimmy K2 something, but now it's uh K3. It's much bigger than before. The previous was 1 trillion parameters. Now, we have almost three uh trillion parameters. And uh it is very very competitive and in this particular arena benchmark this is for front- end coding. You see it keeps the first place and not just the first place. You see how much better it is in comparison with others. So yeah this this is a great success. It's coming from China and it is open weight and uh the pricing uh three for input and $15 per output million tokens if you use it uh via open router. But this model is definitely on level with the best OPUS models. Okay. Uh next. Oh, sorry. This is the shameless plug to my channel. So I have 7,000 subscribers, close to 300 videos. The name of the channel is Left Selector. I always provide links uh for the slides on GitHub and on uh Google Drive. And uh please uh write some comments uh tell me from which country you're watching it and what questions you may have. and then I will maybe feature in the next um next Vulcan and Mojo. So you know that for about 15 years Nvidia had absolute monopoly as far as uh deep learning because they had this CUDA software uh to do uh matrix calculations and uh now we have uh competitors re real like one is Vulcan. So what Vulcan is um it's u a low-level API to do basically the same thing matrix operations. It's coming from a gaming community. Uh so it allows users to uh so you know in games you have graphical games you have this shaders which uh draw um this game but the core operations are basically the same. And these are matrix operations. So you can use it uh as a low-level back end uh to do LLMs. So and because of gamers gamers run on all kind of computers, all kind of CPUs. So it's not only Nvidia, right? Uh so suddenly uh using this Vulcan engine you can run uh LLMs on any platform and do it very very effectively. Um another approach is Mojo. So Mojo is a new programming language and uh uh it is uh coming from Chris Latner's uh company and Chris Lap Latner is a legendary person who uh created uh uh compilers u LLVM and and then you know C lang for example for C language and uh many other projects but Mojo uh well swift programming language for Apple. So Mojo uh is amazing. So first of all it's very simple syntax. It's basically Python but it runs with a speed of C and it runs not only on CPUs but also on GPUs. So you don't have even to think about uh what hardware you have. And they just released version 1 something. So they're getting into production. When I say release, it's in beta still in beta version, but it's coming to production. Uh so you uh can write code which is uh uh absolutely like Python uh but it runs on any hardware and they uh demonstrated it uh on uh in March on Nvidia conference running um on latest Nvidia black wall hardware and basically be absolutely on the level with Nvidia CUDA. Uh so much as Nvidia own handtuned CUDA performance on its own flagship chip using fraction of the code. Uh yeah uh CUDA had about 3,000 lines of code and Mo had only about 700. So it's like 1/4 and doing the same thing. Now Mo's company modular was just acquired by Qualcomm for about $4 billion. Now Qualcomm is a company which designs chips for mobile applications and um some other stuff but this is a big company they don't produce their own phones but they design chips and uh okay global semiconductor okay San Diego California designs and sells Snapdragon processors 5G modems and connectivity chips okay so Vulcan is a crossplatform graphics and compute Created by Kronos group as a modern successor to OpenGL first time released 10 years ago. Kronis host Vulcan core ecosystem on GitHub. So they have multiple GitHub repositories like Vulcan docs, Vulcan headers, Vulcan loader and uh you can go to GitHub, Kronos.org. This is like an entry point to all this uh world. Okay. Acronis group is a nonprofit member-driven standard uh creates standards like OpenGL, WebGL, Vulcan, OpenCL and so on have about 150 180 member organizations and located in Oregon uh US. [snorts] Uh next um Llama CPP versus Lama. Uh just a [snorts] reminder what is Llama CPP? Uh when Meta was only thinking about releasing their first open-source model, Llama, um um they released it like temporarily they they removed it. It was what was it 2 years ago or was it 3 years ago? It's already um it was in March. Um so what what happened uh uh one person from Bulgaria, Gregory Geranov, this is GG in this uh GGUF um standard for um LLM model files. Uh she created a program in C++. he wrote a program which can run the model and then later he created this format and it became very very popular. So llama CPP [snorts] is a a project which creates uh this executable uh which um used to be also um used for fine-tuning the models. Now they mostly doing inference but it runs on different hardware and recently they added ability to run using the Vulcan as an engine. So and suddenly because of Vulcan it runs everywhere right. They also created web UI uh so you can run it locally very comfortably and uh so on Mac for example on Mac Apple silicon you can do it and this is how easy to work with it. So you do brew install llama cpp. This is on Mac. You use brew installer and then you can do llama cli minus m and provide the path for your gguf file with the model and that's it and it will run it. Uh well you can provide uh many more options because this uh uh CLI utility it has many command line options uh to customize your run. It also provides the server llm server and you can give it the model or just refer on hugging face and it will download it and use it from hugging face and if you run it as a server by default it's using port 8080 so you can go here and you will see web UI and work with your model. Okay. Uh so this is how you run llama cpp with vulcan engine. So again llama cli minus m provide the model and then minus ng gl 50 n is the number of layers you want to outsource to vulcan in this case you say 50 layers okay and this is how you do it from python pip install llama cpp python from llama cpp import llama and this is your model and this is how you'll be outputting and u yes and this is uh example how you run it from um bash using curl command, right? Um Alama versus Llama CPP versus VLM. Uh Alama is very popular, very userfriendly, very easy to use. So you just download it from the website and do Alama run and then give it the name of the model. Uh Llama CPP, I just described it to you. It's a low level. In fact, all Llama uses Llama CPP under the hood. Um, so it's a workhorse. Extremely portable, GPU friendly, CPU friendly, great with a graphic quantization. Okay, VLM also another langu um runner to run the models, but it is really good for production when you have long context or you need parallelism. Right. So for local tinkering Alama or Lama CPP is the best for multi- aent long histories multi-user VLM is more effective. And here again about modular and about their runtime they have the their own runtime called max. So it's called modular accelerated execution max inference platform. It's about half a million lines of code and it's uh almost completely open source and I I believe they promised that uh this fall they will outsource uh um open source almost everything uh so modular is uh we're talking about uh max probably not mod not the language but the runner the part which works with large language models um yeah this is what I was before modular booth at GTC 2026 in March. They showed how to do live GPU programming with the latest uh black wall from Nvidia. Um port something and Deepseek V3 and Flux 2 image generation running. Okay. Next, how to use Mojo from Python. So Mojo is very similar to Python but it also if your main application is in Python you can uh uh use pieces in Mojo. So what what you do uh you see from standard Python uh this is the mojo file you see with extension module and you see how it is similar to Python the syntaxes right. So here you define a function which just adds uh two numbers a and b right and then in uh python let's say main.py py you import mod importer import math mode and then you print math mode add two numbers right so what happens the first time when you run the python code it will read this text file the modu code and it will compile it and then the next time it will be already using compiled version and just run it very fast so you can have fast running of LLMs from Python using code in module. Okay, so Max modular accelerated execution AI inference platform high performance from modular lets you build optimize surf models G GPUs CPUs allows deep customization and mostly open source. Okay. Uh lazy graph rag is 25% cheaper. Uh usually when people do rag they first take all their data and they stick it in the database and then they [snorts] use it. But lazy means that you don't process everything up front. You process it as you need it uh during inference. So at query time it combines vector search with iterative deepening uh and uh like overall it reduces indexing co cost so it is cheaper to use uh fable 5 orchestrator and GPT cordex worker. So this is very interesting YouTube video. Uh the author tried different combinations of different models. uh for example it tried a fable uh by itself or fable with cloth us or so he found that the best result he gets when he used uh clo fable as orchestrator and gpt codex worker somehow it produced better result and in fact this is a common theme there are many many authors I've seen they claim the same thing uh esq Pine cone. So pyon is a big uh vector database expensive high performance and nobody today is using it. I mean not nobody's using it but people use it less and less because there are other cheaper [snorts] and simpler solutions. And for example, SQL light um SQL light uh it installed on like billions uh uh places is a 25 year old database or yeah on billion devices uh and uh it is standard SQL it can also use vectors if you you need semantic search right and you you don't need pi pine cone I mean maybe for some extreme cases uh you may need like very high performance performance of pyone but for everything else you simply don't need it uh company yeah and by the way there is a version of SQL light which was already rewritten in rust for high performance [snorts] uh yeah for local setup eliminates Netflix latency slashes builds and so on so if you have your own project just use SQL light or maybe posgressq if it is more evolved it also supports vectors you don't need pine pine cone Okay. Uh, OpenAI released their uh models uh for voice. There is GPT live and GPT life mini. And they are very very fast. So they have full duplex. So they can listen and talk at the same time. Very rapid voice responses and high quality. This is big achievement. How to summarize YouTube videos on Mac OS? Uh uh people ask me how I do those things. So I see a lot of videos but I don't watch them. What I do I copy the URL and then on my Mac I I have the settings. So um okay let me explain. So I go to system settings then keyboards and then text replacements and then I press plus to create a replacement. And um when I press comma comma s it will be replaced with the following phrase. Please make a short 120 words plain text summary of this. So I go into Google uh Gemini uh chatboard. I press comma s and then I press space and this uh fills in this phrase. And then I press commandV to paste the URL of the video. And then I press enter. That's it. So it literally takes like couple seconds to do it with this uh comma comma s uh defined. And uh yeah I use it all the time. Very very effective. Instead of watching the whole video it generates very short. And uh same you can do you don't have to provide the video URL. You can paste some text which you want to summarize. you can provide uh link uh to uh some article uh whatever it works it's it's it's a very good trick um Docker Swarm uh versus Kubernetes uh Kubernetes is still used uh so Kubernetes is a way to run uh uh jobs uh on multiple computers uh using Docker uh uh but you see Kubernetes is uh kind of heavy and for most of the cases so in this uh video they claim about 85% of cases you don't need kubernetes you can use uh docker swarm and docker swarm is a docker's own built-in container orchestration system so it lets you manage cluster of docker engines so you just enable swarm mode in a docker node and together like you can use it as a as as uh swarm. Now another thing web assembly versus docker. So docker is pretty heavy and web assembly can create very small very fast containers which loads in memory in like millisecond and uh it becomes very very popular because you can start many containers like that. It just become very practical. Major platforms like Cloudflare Akami are already deploying WASM at massive scales. So while docker remains dominant for heavy stateful jobs like databases was becoming the go-to standard for edge computing serverless functions and so on. Okay. Um hyperspecific marketing this is by Julia McCoy. Uh what she's saying is that generic marketing message uh doesn't work as well as highly targeted specific message and the more specific you can make the more focus on the specific person who reads this message the better the result will be. Uh next 11 Labs uh now at 600 million annual um recurrent revenue which is huge. Uh so 11 labs was growing very fast. Just a reminder this is the company which became famous by cloning voices. So if you want uh text to voice text to speech this is the best system and it can work in multiple languages. It provides high quality and it can actually copy your voice. Another interesting company is Leia which was rebranded uh into Liora [snorts] and what it is it's a startup originally from Europe from Stockholm in Sweden but now also in London in New York. It's already about uh 650 employees and growing very fast and uh yeah they providing it's a platform for legal work. Uh very successful company growing very fast. Okay. Uh second brain wiki. Uh so Wesroth I always watch all his videos. He showed how he used Andre Karpath's idea of obsidian wiki. So uh this is uh basically bunch of markdown files which are text files in markdown format and they're interlin and uh he created uh his own version of it and he made a video explaining how to do it. So this is his uh structure of directories he used and he just asked AI to create this uh storage and it did. Um we actually tried to do similar thing with uh um one change. We asked instead of generic markdown files to use uh open knowledge format from Google which are the same markdown files but they also have a header portion at the top like a YAML. So it's kind of similar to how skills um are constructed. And it took about half an hour to create a big uh knowledge system. Um so yeah we used um cloth fable to do that. So yeah it's it's a good approach. So you don't need vector database. Uh you you you don't need anything heavy and it is basically free and it's based on uh text files on markdown files on OKFiles. Okay. uh self-improving AI agents uh evolving the hardness not the model. Uh yeah, so this is a good video which explains this approach where as you work with the agent agent iteratively modifying the harness itself. Well, the simplest uh approach I've seen is to modify the claw.md file, but you can make other other changes as well. Okay. uh /workflow command in cloud code and uh what it does it allows you to use uh multiple agents uh and to orchestrate the very complex uh work. They're running a synchronously from primary session thread. Uh developer must actively manage their token budgets. Yeah. because if you do that uh you may um get a very big bill if you're paying for tokens but uh it's a very effective way to do things fast. AI helps quantum computers. So Q day is when quantum computers can break modern encryption and people estimate that it will happen in about 3 years. Uh so question how many cubits needed to achieve that and the estimate was 20 million but now it's actually was dropped 40 times to about half a million because of using additional technologies not just quantum but uh actually using AI and uh Google Cloudflare and IBM have moved their postquantum security deadlines to 29. So another problem may be that immediate threat of harvest now decrypt later. Okay. Next complete uh compete mode technique. So this uh video is about uh using AI for marketing um claude uh using clot for marketing and the idea is to start multiple parallel sub agents uh for speed and efficiency separate blind judge agent evaluates their final outputs and select refine the winning winning version. So we've heard this before. This is just about marketing. Satya Nadella reverse information paradox. Yeah, this is really paradox. So what he is saying that when companies using AI, they taking a risk of giving away valuable property knowledge along with the paying for AI services. uh well for corporate accounts companies claim that they will not use this knowledge to train the models but maybe they do under the hood. So it's ironic to hear this from Microsoft because Microsoft itself is engaged in advancing AI and collecting user data through tools like Copilot. Anyway, um how to use codeex. Yeah, I really like this video by Matthew Berman [snorts] and uh yeah, you see he made picture like Indiana Jones and uh using um GPT using a codeex like a treasure [snorts] and uh okay how to strategically match task difficulty with correct GPT 5.6 Six model soul uh not soul it is [laughter] it is soul like sun terra and luna adjustments to thinking okay how to effectively cross manage delegate tasks using parallel codex threads clean outdated rules and so on so this this is a good video if you want to use uh codex with the GPT models this is a very good instructional video okay open AI charg work an autonomous agent that integrates into apps like Gmail and Slack. Okay, so you you you see the picture. It turns high level goals into finished outputs like documents, deck sites and so on. Powered by GPT56 can perform computer use via CH GPT desktop app. So in um January Claude um I think pioneered like with their claude cowork app for Mac and now we have uh similar thing from uh CH GPT. Well actually everybody now is creating desktop applications. Okay. Anaconda has acquired Kilo code and Kilo code is open source uh uh platform. uh it used by more than three 3 million developers and uh they can uh use it in VS code, jet brains and CLI and they can use more than 500 models and so so on. So it's it's a big system and now Anaconda uh bought them. Anaconda they're from Texas. It it used to be a startup called Continum Analytics and then they were renamed and they created Anaconda packaging for Python but now again they're getting into AI heavily. So for now nothing changes. Kila remains independent with continued open development but they will probably use the technological advances from Anaconda together with Kilo somehow. Uh, Sunno uh, so Sunna is a platform to generate music and it was hacked you see and hacker access soon internal source code infrastructure by infecting an employee with a worm that stole GitHub and cloud credentials. So what they found as a result that Sununa scraped a lot of music and lyrics from sites like YouTube music, DA genius, stock music libraries and hundreds of thousands of podcasts via SS feeds. Okay. Uh Versel Eve opensource file system first AI agent uh framework. So this video explains how it works but advice is that you create a structure of folders for your agents uh to make to streamline the work of the agent. So you have the main agent TS file you have instructions and then you have those folders skills tools sub agents channel schedule and sandbox. So video walks you through all these folders and how to use them and why this system of folders is effective. Uh next entropic rewrote bun into rust. Yeah, this is very interesting. So uh bun is uh kind of like compiler or running um runner for JavaScript. um it actually can u convert TypeScript into JavaScript and run it or it can run uh TypeScript directly. Well, under the hood, it still converts it into JavaScript, but it does it very very fast. And uh so it was originally written in Zigg and Zig is a low-level language. Uh and uh Antropic bought bun and then they re rewritten it uh from zig to rust. They did it in the early May. It took 11 days using 64 parallel cloth agents. The simulated cost if you would pay for those tokens will be like $165,000. So they ported half a million lines of code, resolved 128 long-standing bugs and reduced the binary size. So it's a great success. And uh another example how people nowadays rewrite uh applications into Rust. Rust is very fast and very reliable. Okay. Uh very good GitHub repos. This video lists those 18 repositories. Uh there are some repositories for education. How to work with the local photos with PDFs, secure file sharing, whiteboarding, notion like self-hosted versel APIs, cloud infrastructure or llama and index of indexes. Very very useful links. Uh just go here and uh explore. Okay. Web assembly was 3 is breaking JavaScript 30-year browser monopoly. So uh now uh what's the benefit of wasum web assembly? It runs in browser. It can run much faster than than JavaScript because you can use languages like C++, Rust and Go. And uh they also in the most recent uh version introduced uh native garbage collection which helps languages like Java and Kotlin if you want to use them in W was uh Wasum is moving to serverless environments. So you can run was containers in the browser but you can also run them by themselves uh on on Unix. And this becomes uh very very popular. I already spoke about it directly competing with heavy Docker containers. Okay. Uh Xbox planning to cut uh 50% of their jobs. This is pretty scandalous video. Uh so what he describes is that Xbox actually has 14 layers of management in some divisions. 14 layers. So they planning to flatten the infra this the structure of the company and remove a lot of people in the process. Okay. Uh next video viewing AI as a tool for employee replacement backfires. So companies which fired a lot of people now hiring people back and but using AI for amplification uh to making employees more powerful actually uh this is a success pattern okay Gartner stud reveals that 80% of companies deploy uh deploying a AI and cutting stuff these layoffs fail to improve financial returns and so they they will be rehiring people okay these are the layoff of stat statistics. So you see Microsoft uh cut about close to 5,000 people. And uh this is me as usual and thank you