Vollständiges Transkript anzeigen (4.171 Wörter)
artificial intelligence uh weekly updates every Friday at 2:00 p.m. Eastern time New York time. Today is Friday, August 28. Lots of update as usual and the epigraph for today reviewing and validating is the new bottleneck and we will discuss it later. Okay, these are uh leaderboards of nothing unusual as you see uh blue is Claude, red is Gemini, yellow open AI and green is open source mostly China. So you see blue still uh dominates. Claude is still the best model. Although we have here Kimmy which is Chinese model, Muse Spark which is from Meta and uh CL JLM53 Flash GLM53 Max. Okay, Quen. Where's Quen 3.8? Why I don't don't see it. Um anyway, let's go to the next slide. uh this is artificial analysis intelligence index which is average cost uh per completing tasks. So you see that uh on the right uh cloth uh models are very expensive up to $3 something for Fable and on the left are cheaper models but these are pretty capable models like for example Deepseek V4 Pro is very good models only 27 cents which is more than 10 times cheaper and if you go to Flush which they released uh recently it's only 11 cents. It's not included here, but it's really really affordable. And even more than that, uh you can run it locally uh if you have a good computer with good GPU and memory. Um this is a big thing. So uh Matthew Berman made a review this video which I recommend to watch. So this graph is from Versel. Verscell is a platform where people do VIP coding and then they can actually host their application there on Versel and uh they can use different models and uh the statistics for June was that open models were only 28% and mostly were closed models like open AI or Claude or Gemini but look at what's happening in August 2 months later open models more than doubled. So you see we're comparing this point. So yellow is closed and uh blue is open. Compare it with this on the right. So you see open models are taking over. So American companies actually starting using Chinese models more and more and this is a very strong uh trend. Also what's happening uh people download Quen models uh by millions and uh they uh because it's open source they can change those models they can tune them and you see 151,000 uh derivatives of Quen models currently in existence. Oh this is uh downloads monthly Quen model file downloads 40 million that's what what's happening. So Chinese open source models are becoming very very popular and this is a very strong trend. Nvidia AVO agent improves clo OPUS 5 performance on AR AGI 3 from 30% to 100%. Which is unbelievable. So just to remind you uh ARC AGI means abstraction and reasoning uh corpus for artificial general intelligence. Abstraction and reasoning. So the first version so currently we have version three but the first version were puzzles puzzles like graphical puzzles which was very easy for a human to solve but models like have real real difficulty. Then uh they learned how to solve those puzzles. So they created version two and uh then version three which is uh not just puzzles but it's interactive. It's more like a playing computer game and uh u regular models uh you see for example cloth us 5 30% GPT56 soul which is the top of the line for openi only like less than 8% clo oppus 4.8 3.6 So they're not performing well. But what Nvidia did, they created an agent which was solving those puzzles and they achieved 100%. Which is uh like unbelievable which shows again that agent are performing much better than just regular models. So here some links about arc and arc price. They actually have $2 million prize, but uh this Nvidia couldn't participate because it's not a model, it's a an agent. Okay. Uh next, Deepseek Flash gets vision, right? So, Deepseek released version 4 flash. It was uh end of uh July and now they added vision. So you can provide images and this curve is absolutely amazing. So here's what's happening. Deepseek harness was released on August 13th and it achieved actually it's not 191 it's currently it's 200,000 stars and you see that they achieved 100 uh uh thousand stars basically in less than 2 days and then it kind of slowed down but still it's the very very fast uh progress a lot of interest so this table which I compiled shows uh the fastest project on GitHub. So the fastest fastest is called claw code and what happened on in the end of March uh the code of clot code tool the CLI was leaked and people took this code which suddenly became in the open and they rewrote it and so this is claw code is the most popular uh version of it and they achieved hundreds thousand stars in one day. So basically these are stolen good good goods right but then deepseek is official release by by deepseek and you see they achieved 100,000 stars in less than 2 hours next contenders we have open claw uh about 2 days her agent about a week oh no no sorry 7 to 10 weeks okay GPT I'm not sure why they listed in this order uh current stars. Okay. Lang chain uh defy uh N8N Alama Llama CPP. Okay. Uh next, OpenAI took their codeex harness and they released it in open source. This is a big thing. Uh in fact, what happened? So we have Gro uh leaked and then released the Gro build. It was in February. Then clo code was leaked like now open AAI officially released their their stuff. There are a lot of harnesses which are now in open source and uh can can be used uh like and you can take the harness. Hardness is not a model. Harness is code. So you can take it and you can modify it. So you can ask cloud code to modify the harness to your liking. uh OpenAI uh merges uh chat GPT into Codex into the Chad GPT app. So this is the interface of Chad GPT and you see it now has Codex and if you click it has two um either work or Codex for coding. So Codex now uh is not like a separate application. It's more like a choice in chat GPT application. Now, Jalapina uh OpenAI chip uh it performs very well in tests. It uh better than Nvidia stuff and they're planning to start deploying it in the end of this year. Okay. Oh, this I compiled a list of 52 harnesses which are open open code and some of the known one Codex CLI PI which is like a minimal one open code clock code Grog build harness her agent open claw of course lang chain autogen open AI agents SDK GPT crewi Microsoft agent framework for multi- aent lang flow n uh open daven open manus I I just listed the most famous one but you see how many so now if you want to create your own agent uh you have a choice and of course you can use clo closed ones like for example you can use clo code as a backbone of your agent and then add uh skills and um MCP uh and scripts or workflows or whatever you need on top of it. Okay. uh in China 2026 World Humanoid Robot Games and they uh it's a big event what I really like this robot which uh is beating the record set by a human like Usain Bolt in 2009 uh had 9 seconds 58 9.58 seconds so less than 10 seconds and uh this robot is beating this record and they actually has beaten many many records on those games. It's a lot of fun. A lot of videos on YouTube. I recommend you to watch. Okay. Some rumors. Uh Google has confirmed Gemini 4 pre-training uh started. Uh ox alpha on open router. This is uh anonymous model. Nobody knows what it is. It is performing very well. U SSI. This is company. They promised their release in August. It haven't happened yet, but you know, since creation, they were very very secretive. Uh so we don't know what it is. Uh US agencies warn of an active campaign targeting exposed industrial controllers. So this is an example of industrial controllers semens and uh uh they were under attack. Uh next uh okay this is plug for my channel. Name of the channel it's my name left selector. I have actually more these numbers needs to be updated but more than 7,000 subscribers and 310 videos and now I'm making short videos. If you go my channel and click shorts you will see some short videos. They are automatically generated uh but fun fun to do. Uh download slides for yeah slides are posted to GitHub and to Google Drive. The links are under the video and please provide feedback to the video. Please ask if you want me to talk about certain topics. Uh please let me know. Okay. Entropic added cross chat and co-work memory controls with editable deletable topics and uh opt-in setting for sensitive topic memory. So this is important for enterprise uh settings. Uh European AI act transparency rules uh finally took effect this month. So users must be informed when interacting with AI. Certain synthetic media defects require labeling and so on. Quen okay this is yet another model from Quinn 3.8 flash next. So usually when people talk about 3.8 they either they talk about the 27 uh um uh billion parameter model but this is different. This is 125 billion core parameters and 51 billion engram embedding parameters. It's so it's only 6 billion active parameters. Context length at 262. It actually requires a lot of memory to run uh like uh maybe about 150 gig of memory. So it was just released on hugging face. It's open weights. Uh what's interesting that this open model is better than clo oppus 4.6 marks on benchmarks. So open model models now are very very close to the top-of-the-line frontier models. So it can be used via open code and open router. And as you can see it's very inex inexpensive. So here is an example. The model was coding for 44 minutes and the total cost of the whole development was uh about half a dollar. So it's very very cheap, dramatically cheaper coding agents workflow including whatever. Uh so this is quen 38 flash next uh kil linear cheaper long context. So it's a hybrid transformer architecture. It's a Kimma delta attention KDA layer use recurrent linear style attention. Okay, so uh the transformer it's not a simple transformer as you can see but much smaller cache lower uh memory requirements. So it's a very good progress with that. Uh Gemini enterprise for legal all major providers have uh plugins or models or packages. So now for for legal work. So Google Gemini enterprise for legal suit of AI skills and agents for legal teams. Okay. Next uh robotics and VA which is vision language action. Now every time uh I talk about this. So what's happening now? Uh what are the trends? Uh scaling highquality training data uh research moving toward memory and temporal reasoning. So not just static static pictures but how things develop in time stream uh pi released on August 26 adds parameter free streaming okay applied VA for specialized environments uh so for example uh greenhouse right so you growing uh vegetables and robots uh like servicing those vegetables uh combine language VA models 3D plant reconstruction action human oversight for safe robotic plant care tasks. Okay. Uh the main remaining bottleneck is robust generalization. A robot can still fall uh when lightning object pose surfaces or when something changes uh robot suddenly can just fall or do something wrong. Okay. Open AAI launched Apple messages plug-in for chat GPT like you know like open claw her or whatever they can use different chat platforms. Uh but here's open AAI uses uh I messages which is Apple messages. You see this combination okay perplexity plus Nvidia local agent. This is big. I love perplexity. Like frankly when I use AI probably 90% of my requests I use perplexity well except for when I'm doing coding. Uh so now what perplexity have done uh they joined with Nvidia and if you have a reasonable computer like for example you can use uh Nvidia DGX Spark or you can use just a regular computer with something like RTX 3090 or 4090 with 24 GB of RAM which is nothing special just a regular desktop and you can use local models. So what they give you they give you application which runs models which can run on this hardware and good enough for most tasks. Um so it's different from Alama for example. Alama uh it supports thousands and thousands of different models. It's a run engine for models. This gives you it's basically agentic work uh for local work and I guess this is a new trend now and for most people this is all they need because uh no coding is needed you just take this and you just run it and it's good enough okay habit hooks avoid agent slope uh so these are basically skills uh you see tool install habit hooks uh which help you help your agent your coding agent to uh make your code better to avoid what's called uh slope uh cohesive modules clear naming separation of concerns and so on this I guess it's a very good thing now free token open local inference engine so GLM 5.2 2. This is the model at 15 tokens per second on 96 GB RTX Pro 6000 or like with quen 36 reaches 39 tokens per second. Okay, this is the GitHub. It's uh open and they work with Nvidia with CUDA uh so best fit recent Nvidia hardware ample RAM and sustained workloads. So this is uh software which allows your models to run faster on Nvidia hardware. Okay. Mixed bread toast one AI search sub agent. Uh so this is uh how how it works. Decomposes questions. Run multiplestep searches. Inspect sources. Returns compact evidence packages. Uh you see it's not free but it's good when you doing search it's actually good to use a tool like that. There are also some free tools but this is high quality uh tool which is not free but good uh open bot self-hosted AI co-workers uh platform for running isolated AI co-workers in alpha stage MIT license so it's open source but it's only starting each coworker can receive separate browser files tools credentials and they work uh all work together okay Uh next unsloth efficient LLM training. So you probably seen uh the word unsloth uh multiple. So it's unsloth.AI it's a project on GitHub. So this is sloth. So this is animal which [snorts] climbs the trees and it's very very slow slow moving tree dwelling mammal from Central and South America. So unsloth is a a way to do LLM training and to do it fast. open source toolkit that speeds up and reduces GPU memory usage. I found it uh three years ago. Efficient Lora and Qura fine-tuning uh requires less memory and so on. Has many benefits. I just wanted to provide the picture because sometimes people have difficulty remembering the name unsloth. So sloth is slow, unsloth is fast. Uh next uh four times more context from the same GPU. So modern models use less uh key value cache. Uh when you have attention layers in transformers each layer has cache and uh but modern models they don't put cache in every layer. So you see modern models have hybrid architectures. For example, Quen uses uh 48 gated Deltaet layers and 16 full attention layers. Only these attention layers uh have uh KA cache potentially cut and the rest the 48 they don't have. So you don't need as as much cache. That means that you don't need as much memory and you can process longer context. Uh another example is Gemma 3. it interleaves five uh sliding attention layers with one global layer. So cache growth become slower. Same with the GPTO OSS and so on. Okay. Or 1.5 by deep reinforce. So it's a self-eing AI open weight agentic coding models small 9 billion dense 35 billion or 397 billion mixture of experts. Actually 35 billion is also a mixture of experts. Sorry. Uh so the uh 397 billion close to clo oppus 48 on benchmark. So it's pretty good and uh yeah so you have uh variance and it is open 8 open weight models. So this is good uh hiking phase github quen 3.827b 827B. You can run it on a regular uh video card, the video GPU like RTX3090, which is not new. It's pretty cheap and affordable 24 GB card. And uh you can also run it on DGX Spark. uh software uh tuning raises single user throughput from uh 46 to 133 tokens per second which is very good. Optimized batching reaches,000 tokens per second across 64 request and uh so what they do they do uh certain tensor compute f16 delta state quantized embedding improve speculative decoding and prompt based uh copying. Anyway, so this new model is better. It's faster. Uh there is a quantized uh versions like 4bit, 8 bit. Uh quality validation matters in early kernel produced garbage despite uh uh excellent benchmark throughput. Okay. So this is this is very good new models which you can run locally. Uh, Turbo Fieldfare SSD streamed LLM runs Gemma 4 26 billion uh, parameters on Apple silicon with only two gigabyte resident memory. How they do it? By retaining its 1.35 GB shared core and Kache while streaming routed MI experts from SSD. So, uh, they use fast SSD. SSD is like a hard drive, right? Uh uh amazing. So not fast, right? Five six tokens per second which is not fast but it runs on very computer with very small memory and yeah this is good uh SSD solid state transformers. This is not AI but this is just something uh interesting. So you're familiar with modern chargers like this is anchor uh charger for USB and uh internally these chargers use uh high frequency transformers. Right now in uh in regular electronic grids you have those lines which are kilovolts of AC uh currents. Uh so this is uh 60 Hz in America and but uh in order to get to homes you need to lower the voltage from kilovolts to approximately 100 volts. It usually goes in stages and what usually people do people use huge transformers which are very heavy a lot of metal but the new technology you can use something like that you can use high frequency. So you go from high voltage to high frequency transformation and now you work in low voltage and this is example of such device they now being introduced and put in production. So this is new technology. Okay. Um AI evals human taste at scale. Uh short for evolves short for evaluations systemic tests used to measure quality accuracy safety performance artificial intelligence models applications. Uh start evolves from real output and traces not an abstract checklist. Split criteria into top down or bottom up. So top down known task requirements such as format length actionability policy compliance bottom up recurring flow flaws discovering by comparing generated outputs human approved results. Uh so this is kind of goes how you do evals use separate LLM judge passes or sub agents for many criteria. Okay. So th this is like a recipe how to do evolves and here some useful links how to do it. Uh next modern rag system architecture. This is interesting. Uh so I was on interview and they have a project they need basically a rack system retrieval uh augmented generation which handles millions of documents in different formats. Right. And I worked with AI back and forth to create a system designed for that. And uh so the retrieval combines three methods vector semantic search uh BM25 for keyword search and uh type document graph which is basically wiki like we spoke about obsidian wiki using markdown file. But in this case all these three search mechanisms are stored in one place. which is posgress SQL database. Pasgress has plugins for vector search for keyword search and you can also effectively work with kind of a wiki and uh yeah this is very interesting because you get high quality rack system but at the same time pretty simple architecture. uh this is the diagram of uh this system and uh the big part of it so when you have documents if they're electronic and structured already it's a native parser it's easy but if you have let's say PDFs or um slides which you need to scan and examine and extract information from them so this is actually the heaviest part of the whole system okay next uh Alibaba 1 3.0 document to video AI. So it generates up to 30 second videos with native audio from documents. Very interesting. Next uh Claudeex uh loop skill for dual model work. So you have uh you want to use let's say for example claude and codex and you can use cordex loop. So it's a it's a GitHub project. It's open source to uh separate which model is used for which uh purpose. Uh anthropic and house silicon team for custom code. So uh you know that open AAI is working on their gelapino. Google of course have their own TPUs and now anthropic is also working on their in-house uh uh chips. So they are hiring people paying very high salaries uh for people who can work on designing those chips. Uh so custom chips are likely a long-term efficiency and inference cost strategy. Okay. Shyomi I I don't know how to pronounce it. Aluminum AI cube. This is very very in interesting project. So it's a prototype. It's not released uh yet. But how uh people create chips? They have uh they cut uh wuffer like a [snorts] u how to say thin thin slices of uh silicon and then they use lithography uh to uh create like pictures of uh what they need and then they test them and they cut them. Uh but what uh these people have done they create several waffers and they connect them together and they form a cube and only after that they can test and of course uh if one of those slices have a problem then they have to discard uh like the whole cube. So it may be more uh expensive but the benefit is that once you successfully created those this thick cube uh chip then the bandwidth of data transfer inside this chip is much higher. So andomi decided to to do this technology. Okay. Podman versus Docker. So uh docker uh you know it's the the way you create containers and you run multiple containers on the same computer. So you have a server uh like a docker server which runs those um containers. So it's uh rootful by default. So it runs as as a root. You have a docker demon. Uh now podman is a different animal. it made uh to uh be very similar to docker but it doesn't require you to run a demon to to run a root uh user demon. So you can use a regular user to run containers. Uh the syntaxes of those containers are very similar to to docker but overall technology is kind of easier to use. So, Podman supports rootless containers run under ordinary unprivileged Linux user and largely uh Docker compatible. So, if you never heard about Portman, look at it. Now Meta uh cut a lot of their people 10% in May and what they found is that although they started creating more code with AI you see uh doubled feature delivery grew only 36%. and uh under supervised agents which is uh not enough human supervision caused disruptive changes. Technical and security incidents rose 40 uh 40% with incident response time up to 70%. So what they decided to do they decided not to uh fire more people at this point they realized people are still needed. Okay acknowledged slow than expected AI acceleration. Okay. Okay. These are layoffs information for August. You see not many layoffs at all. And this is me as usual. And thank you.
Video
Video
Video
Video
Video
Video
